Skip to content

Specify versions for the Codey models + text-bison

Alexander Chueshev requested to merge ac/fix-vertex-model-version into main

What does this merge request do and why?

This MR sets the versions of the Codey models and text-bison to @001 following the request in https://gitlab.com/groups/gitlab-org/-/epics/11224#note_1546092158.

Initially, we used the PALM_TEXT_MODEL_NAME env variables for such purposes. However, this env variable looks outdated and needs to be updated. The current infra uses the latest models, so I hardcoded the model versions to prevent any unexpected changes asap and have time to update config.py and deps.py.

How to set up and validate locally

The response and logs now contain the new model name with the version concatenated. Example:

{
  "id": "id",
  "model": {
    "engine": "vertex-ai",
    "name": "code-gecko@001",
    "lang": "go"
  },
  "experiments": [
    {
      "name": "exp_truncate_suffix",
      "variant": 0
    }
  ],
  "object": "text_completion",
  "created": 1694119764,
  "choices": [
    {
      "text": "if err != nil {\n\t\thttp.Error(w, err.Error(), http.StatusInternalServerError)\n\t\treturn\n\t}\n\th.hotelsStore.SetHotels(hotels)",
      "index": 0,
      "finish_reason": "length"
    }
  ]
}

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Alexander Chueshev

Merge request reports