Skip to content

Code completions v3 endpoint return not compatible with v2 or multiple suggestions

Right now, the v3 code completions endpoint returns data in this format

{
  "response": "\n\tf, err := os.Open(filename)\n\tif err != nil {\n\t\treturn nil, err\n",
  "metadata": {
    "model": {
      "engine": "vertex-ai",
      "name": "code-gecko@002",
      "lang": "go"
    },
    "timestamp": 1716492294
  }
}

While I understand moving to v3 from v2 should be a breaking change and the input is in a different format, the output from v3 has the problem that there is no longer any room for multiple choices from one request, which we recently implemented for code-gecko to support the IDE plugins. Further, other than the moving around the metadata I'm not sure why we would change the response type from what v2 returns.

{
  "id": "id",
  "model": {
    "engine": "vertex-ai",
    "name": "code-gecko@002",
    "lang": "go"
  },
  "experiments": [
    {
      "name": "exp_truncate_suffix",
      "variant": 0
    }
  ],
  "object": "text_completion",
  "created": 1716493576,
  "choices": [
    {
      "text": "\n\tfile, err := os.Open(filename)\n\tif err != nil {\n\t\treturn nil, err\n",
      "index": 0,
      "finish_reason": "length"
    }
  ]
}

We should update the v3 endpoint return values to be compatible with multiple choices, and ideally just make it match the v2 return value.

Edited by Allen Cook