You need to sign in or sign up before continuing.
Consolidate tokenizer logic in code suggestions
What does this merge request do and why?
Consolidate tokenizer logic in code suggestions.
Code completions via Vertex models utilize duplicate tokenization logic with code completions via Anthropic and code generations. This results in maintenance overhead and surprising bugs due to duplication.
This consolidates tokenization logic into TokenizerTokenStrategy
and
standardize the interface of CodeCompletionsLegacy
. As part of this
refactor, some types are also moved to processing.typing
to avoid
circular dependency.
How to set up and validate locally
- Check out to this merge request's branch.
- Ensure the server boots up normally.
poetry run ai_gateway
- Run a few cURL requests, esp. code completion via Vertex AI.
curl --request POST \ --url http://0.0.0.0:5052/v2/completions \ --header 'Content-Type: application/json' \ --header 'X-Gitlab-Authentication-Type: oidc' \ --data '{ "prompt_version": 1, "project_path": "awesome_project", "project_id": 23, "current_file": { "file_name": "main.py", "content_above_cursor": "def prime_numbers(n):\n\t'\'''\'''\''Generate a list of ", "content_below_cursor": "" } }'
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Tan Le