Test palm models by restricting access with project_id
This MR limits access to the PALM models using project_id.
Use cases - https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/e66bf103c21d88561fb3208aef18a0f79391c272/tests/codesuggestions/test_api_suggestions.py
How to run locally:
-
Run
poetry install
in the project root directory -
Generate a JSON key file for the vertex service account used in the staging cluster. Please, delete it after
-
Update the
.env
file with the following content:GOOGLE_VERTEX_AI_CREDENTIALS="<path to the JSON key>" AUTH_BYPASS_EXTERNAL=false # required false to test this MR TRITON_HOST=host.docker.internal TRITON_PORT=8080 F_IS_THIRD_PARTY_AI_DEFAULT=true # or false FASTAPI_DOCS_URL=/docs FASTAPI_OPENAPI_URL=/openapi.json FASTAPI_API_PORT=5052
-
Run
poetry run codesuggestions
in the project root directory -
Send a request using
curl
to the/v2/completions
endpoint with your PAT.
Closes: #146 (closed)
Edited by Stan Hu