Skip to content

Apply text PaLM API changes

Andras Herczeg requested to merge add-palm-endpoint into main

This MR introduces a third-party text-bison model to model-gateway.

How to run locally:

  1. Run poetry install in the project root directory
  2. Generate a JSON key file for the vertex service account used in the staging cluster. Please, delete it after
  3. Update the .env file with the following content: 
    GOOGLE_APPLICATION_CREDENTIALS="<path to the JSON key>"

AUTH_BYPASS_EXTERNAL=true
TRITON_HOST=host.docker.internal
TRITON_PORT=8080
F_IS_THIRD_PARTY_AI_DEFAULT=true   

FASTAPI_DOCS_URL=/docs
FASTAPI_OPENAPI_URL=/openapi.json
FASTAPI_API_PORT=5052
  4. Run poetry run codesuggestions in the project root directory
  5. Open localhost:5052/docs and your browser and send any requests to the /v2/completions endpoint.

Closes https://gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/-/issues/142

Edited by Alexander Chueshev

Merge request reports