Skip to content

Use claude-2.1 for code generations

Jan Provaznik requested to merge jp-cs-claude21 into master

What does this MR do and why?

If code_suggestions_claude21 flag is enabled, we set explicitly model_name=claude-2.1 for code generation requests. If the flag is not set, we use AI gateway's default model (currently claude-2.0).

Related to #432636 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. In rails console enable the flag 
    Feature.enable(:code_suggestions_claude21)
  2. In AI gateway repo, switch to the branch of this MR which adds model_name support on AI gateway side
  3. Make sure you use your local GDK setup in vscode extension
  4. Trigger a code creation request (e.g. by writing a comment in a source code file) and verify in AI gateway logs that model_name=claude-2.1 is used for outgoing request and incoming response
Edited by Jan Provaznik

Merge request reports