feat: return characters stop sequences for Vertex-on-Codestral
What does this merge request do and why?
We introduced Vertex-on-Codestral model for code completions in Add Codestral on Vertex support for code comple... (#564 - closed). That is now in production (though only enabled for the groupcode creation team and a few other GitLab members).
After gathering the first round of feedback, one of the main tasks we surfaced was adding a \n\n
to the stop sequences for Codestral similar to what we did with Code Gecko.
How to set up and validate locally
Test with this request, where the answer starts with a \n\n
:
curl "http://gdk.test:5052/v2/code/completions" \
-X POST \
--header "Content-Type: application/json" \
--data "{\"current_file\":{\"content_below_cursor\":\"\",\"file_name\":\"src/python-django-app/main/admin.py\",\"language_identifier\":\"go\",\"content_above_cursor\":\"from django.contrib import admin\"},\"stream\":false,\"prompt_version\":1,\"model_provider\":\"vertex-ai\",\"model_name\":\"codestral@2405\"}" \
| json_pp -json_opt pretty,canonical
On main
branch, running a code completion request without the \n\n
should result in a response with a non-empty code suggestion:
{
"choices" : [
{
"finish_reason" : "length",
"index" : 0,
"text" : "\n\nfrom .models import Address, City, Client, ClientAddress, ClientPhone, \\\n Contract, ContractAddress, ContractAddressType, ContractPhone, \\\n ContractPhoneType, ContractType, LegalEntity, LegalEntityAddress, \\\n LegalEntityAddressType, LegalEntityPhone, LegalEntityPhoneType, \\\n Organization, OrganizationAddress, OrganizationAddressType, \\\n OrganizationPhone, OrganizationPhoneType, Phone, PhoneType, \\\n ReasonForCancel, Status, StatusLog, User\n\n# Register your models here.\n\nadmin.site.register(User)\nadmin"
}
],
# other details here
}
On 625-codestral-add-returncharacters-stops
branch, running a code completion request WITH the \n\n
stop sequences should result in an EMPTY code suggestion:
{
"choices" : [],
# other details here
}
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Closes #625