Support Anthropic in code creation
What does this merge request do and why?
This MR implements the Anthropic model provider
and adds an additional model_family
field to select the Code Creations engine (vertex ai or anthropic) when sending POST requests.
For code creation, we use the claude-2
model configured in deps.py
.
The model hyperparameters can be configured in deps.py
or when calling the model via model.generate(...)
.
Please, check the unit tests for more information on how the Anthropic provider can be used.
If model_family
is not specified in the request payload, the gateway will rely on the Vertex AI code-bison
model to generate suggestions.
How to set up and validate locally
-
Configure
ANTHROPIC_API_KEY
in the.env
file. -
Make request to
/code/generations
:anthropic provider
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "\n\nHuman: print hello world", "content_below_cursor": "" }, "model_provider": "anthropic", "telemetry": [], "prompt_version": 1 }
vertex ai provider
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "\n\nHuman: print hello world", "content_below_cursor": "" }, "model_provider": "vertex_ai", "telemetry": [], "prompt_version": 1 }
vertex ai (by default)
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "\n\nHuman: print hello world", "content_below_cursor": "" }, "telemetry": [], "prompt_version": 1 }
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Depends on !406 (merged) Ref #311 (closed)
Edited by Alexander Chueshev