Skip to content

Test palm models by restricting access with project_id

Alexander Chueshev requested to merge test-palm-models-by-project-id into main

This MR limits access to the PALM models using project_id.
Use cases - https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/blob/e66bf103c21d88561fb3208aef18a0f79391c272/tests/codesuggestions/test_api_suggestions.py

How to run locally:

  1. Run poetry install in the project root directory

  2. Generate a JSON key file for the vertex service account used in the staging cluster. Please, delete it after

  3. Update the .env file with the following content:

    GOOGLE_VERTEX_AI_CREDENTIALS="<path to the JSON key>"
    
    AUTH_BYPASS_EXTERNAL=false # required false to test this MR
    TRITON_HOST=host.docker.internal
    TRITON_PORT=8080
    F_IS_THIRD_PARTY_AI_DEFAULT=true # or false 
    
    FASTAPI_DOCS_URL=/docs
    FASTAPI_OPENAPI_URL=/openapi.json
    FASTAPI_API_PORT=5052
  4. Run poetry run codesuggestions in the project root directory

  5. Send a request using curl to the /v2/completions endpoint with your PAT.

Closes: #146 (closed)

Edited by Stan Hu

Merge request reports