Skip to content

Set claude 2.1 as model for slash commands

Jessie Youngrequested to merge
jy-slash-commands-claude-3 into master

What does this MR do and why?

  • The ensure that all tools have a model set
  • Otherwise, they use the model default, which is Claude 3 sonnet if the user has a feature flag enabled. But these tools are still sending json request bodies for the completions API.
  • See code here for default model config
  • We eventually want to migrate these tools to Claude 3, but we want to support them staying on Claude 2.1 for now so we can selectively enroll users into Claude 3 without degrading behavior for existing customers.
  • Resolves #454849 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After
Screenshot_2024-04-09_at_3.05.45_PM Screenshot_2024-04-09_at_3.02.44_PM

How to set up and validate locally

  1. In rails console enable the experiment fully 
    Feature.enable(:ai_claude_3_sonnet)
  2. Test the slash commands: /explain, /refactor, and /tests. All should work and be using Claude 2.1

Also good to ensure that all anthropic prompts have the model option set:

 git ls-files | grep anthropic.rb
ee/lib/api/ai/experimentation/anthropic.rb
ee/lib/code_suggestions/prompts/code_generation/anthropic.rb

ee/lib/gitlab/llm/chain/agents/zero_shot/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/requests/anthropic.rb
ee/lib/gitlab/llm/chain/tools/ci_editor_assistant/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/epic_identifier/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/epic_reader/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/explain_code/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/issue_identifier/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/issue_reader/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/refactor_code/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/summarize_comments/prompts/anthropic.rb
ee/lib/gitlab/llm/chain/tools/write_tests/prompts/anthropic.rb

We can ignore the first 2 serach results because code suggestions doesn't use the AiGateway::Client and sets model headers here

Manual check for MODEL being set in all other tools:

  • ee/lib/gitlab/llm/chain/agents/zero_shot/prompts/anthropic.rb (this prompt works for claude 3 when feature flag enabled so no MODEL needed)
  • ee/lib/gitlab/llm/chain/requests/anthropic.rb (not a tool so not applicable)
  • ee/lib/gitlab/llm/chain/tools/ci_editor_assistant/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/epic_identifier/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/epic_reader/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/explain_code/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/issue_identifier/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/issue_reader/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/refactor_code/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/summarize_comments/prompts/anthropic.rb
  • ee/lib/gitlab/llm/chain/tools/write_tests/prompts/anthropic.rb
Edited by Jessie Young

Merge request reports