Support Anthropic in code creation
What does this merge request do and why?
This MR implements the Anthropic model provider
and adds an additional model_family field to select the Code Creations engine (vertex ai or anthropic) when sending POST requests.
For code creation, we use the claude-2 model configured in deps.py.
The model hyperparameters can be configured in deps.py or when calling the model via model.generate(...).
Please, check the unit tests for more information on how the Anthropic provider can be used.
If model_family is not specified in the request payload, the gateway will rely on the Vertex AI code-bison model to generate suggestions.
How to set up and validate locally
-
Configure
ANTHROPIC_API_KEYin the.envfile. -
Make request to
/code/generations:anthropic provider
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "\n\nHuman: print hello world", "content_below_cursor": "" }, "model_provider": "anthropic", "telemetry": [], "prompt_version": 1 }vertex ai provider
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "\n\nHuman: print hello world", "content_below_cursor": "" }, "model_provider": "vertex_ai", "telemetry": [], "prompt_version": 1 }vertex ai (by default)
{ "project_path": "string", "project_id": 0, "current_file": { "file_name": "app.py", "language_identifier": "python", "content_above_cursor": "\n\nHuman: print hello world", "content_below_cursor": "" }, "telemetry": [], "prompt_version": 1 }
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Depends on !406 (merged) Ref #311 (closed)
Edited by Alexander Chueshev