Code completions v3 endpoint return not compatible with v2 or multiple suggestions
Right now, the v3 code completions endpoint returns data in this format
{
"response": "\n\tf, err := os.Open(filename)\n\tif err != nil {\n\t\treturn nil, err\n",
"metadata": {
"model": {
"engine": "vertex-ai",
"name": "code-gecko@002",
"lang": "go"
},
"timestamp": 1716492294
}
}
While I understand moving to v3 from v2 should be a breaking change and the input is in a different format, the output from v3 has the problem that there is no longer any room for multiple choices from one request, which we recently implemented for code-gecko
to support the IDE plugins. Further, other than the moving around the metadata I'm not sure why we would change the response type from what v2 returns.
{
"id": "id",
"model": {
"engine": "vertex-ai",
"name": "code-gecko@002",
"lang": "go"
},
"experiments": [
{
"name": "exp_truncate_suffix",
"variant": 0
}
],
"object": "text_completion",
"created": 1716493576,
"choices": [
{
"text": "\n\tfile, err := os.Open(filename)\n\tif err != nil {\n\t\treturn nil, err\n",
"index": 0,
"finish_reason": "length"
}
]
}
We should update the v3 endpoint return values to be compatible with multiple choices, and ideally just make it match the v2 return value.
Edited by Allen Cook