Skip to content

Support Anthropic in code completion

Alexander Chueshev requested to merge ac/suppport-anthropic-code-completion into main

What does this merge request do and why?

This MR adds support for the Anthropic provider when completing the code. To activate the Anthropic provider, set the model_provider field to anthropic when sending requests to /v2/code/completions. Otherwise, the gateway will rely on the Vertex AI code-gecko model to generate suggestions.

To complete the code with Anthropic support, we use the claude-instant-1 model configured in deps.py with 128 output tokens. Other model hyperparameters can be configured in deps.py as well or when calling the model via model.generate(...).

Please, check the unit tests for more information on how the Anthropic provider can be used. Please, note that when choosing an Anthropic provider, we do not have any pre/post-processing. We can add additional logic with any follow-up MR.

How to set up and validate locally

  1. Configure ANTHROPIC_API_KEY in the .env file.

  2. Make request to /v2/code/completions:

    anthropic provider
    {
        "project_path": "string",
        "project_id": 0,
        "current_file": {
            "file_name": "app.py",
            "language_identifier": "python",
            "content_above_cursor": "def hello_world",
            "content_below_cursor": ""
     },
        "model_provider": "anthropic",
        "telemetry": [],
        "prompt_version": 2,
        "prompt": "\n\nHuman: def hello_world"
    }
    vertex ai provider
    {
        "project_path": "string",
        "project_id": 0,
        "current_file": {
            "file_name": "app.py",
            "language_identifier": "python",
            "content_above_cursor": "def hello_world",
            "content_below_cursor": ""
     },
        "model_provider": "vertex-ai",
        "telemetry": [],
        "prompt_version": 2,
        "prompt": "def hello_world"
    }
    vertex ai (by default)
    {
        "project_path": "string",
        "project_id": 0,
        "current_file": {
            "file_name": "app.py",
            "language_identifier": "python",
            "content_above_cursor": "def hello_world",
            "content_below_cursor": ""
     },
        "telemetry": [],
        "prompt_version": 2,
        "prompt": "def hello_world"
    }

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.

Closes #310 (closed)

Edited by Alexander Chueshev

Merge request reports