Support Anthropic in code completion
What does this merge request do and why?
This MR adds support for the Anthropic provider when completing the code.
To activate the Anthropic provider, set the model_provider
field to anthropic
when sending requests to /v2/code/completions
.
Otherwise, the gateway will rely on the Vertex AI code-gecko
model to generate suggestions.
To complete the code with Anthropic support, we use the claude-instant-1
model configured in deps.py
with 128
output tokens.
Other model hyperparameters can be configured in deps.py
as well or when calling the model via model.generate(...)
.
Please, check the unit tests for more information on how the Anthropic provider can be used. Please, note that when choosing an Anthropic provider, we do not have any pre/post-processing. We can add additional logic with any follow-up MR.
How to set up and validate locally
-
Configure
ANTHROPIC_API_KEY
in the.env
file. -
Make request to
/v2/code/completions
:anthropic provider
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "def hello_world", "content_below_cursor": "" }, "model_provider": "anthropic", "telemetry": [], "prompt_version": 2, "prompt": "\n\nHuman: def hello_world" }
vertex ai provider
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "def hello_world", "content_below_cursor": "" }, "model_provider": "vertex-ai", "telemetry": [], "prompt_version": 2, "prompt": "def hello_world" }
vertex ai (by default)
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "def hello_world", "content_below_cursor": "" }, "telemetry": [], "prompt_version": 2, "prompt": "def hello_world" }
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Closes #310 (closed)