Migrate AI features to Claude 3.5 Sonnet
Description
The objective is to migrate from Claude 3 Sonnet to Claude 3.5 Sonnet across all relevant platforms and tools within the organization. This migration aims to leverage the improved performance and capabilities of Claude 3.5 Sonnet, which has demonstrated superior results in initial evaluations.
We successfully migrated tools to Claude 3 models during %17.0 milestone, so the upcoming work should follow a similar path to Migrate Duo Chat tools issue.
Tasks
To scope out the work, we should consider the following AI features based on Unit Primitives that are grouped together. The following Duo Chat features gives us a sense just for Duo Chat
Duo Chat AIGW Agent:
-
ai_gateway/chat/container.py
=> Will be captured as a different issue.
Duo Chat tool:
Chat Tools
-
ee/lib/gitlab/llm/chain/tools/ci_editor_assistant/prompts/anthropic.rb
- CI Editor -
ee/lib/gitlab/llm/chain/tools/gitlab_documentation/executor.rb
- Gitlab Documentation -
ee/lib/gitlab/llm/chain/tools/epic_reader/prompts/anthropic.rb
- Epic Reader -
ee/lib/gitlab/llm/chain/tools/issue_reader/prompts/anthropic.rb
- Issue Reader
NOT PUT INTO PRODUCTION YET
-
ee/lib/gitlab/llm/chain/tools/merge_request_reader/prompts/anthropic.rb
- Merge Request Reader -
AI Gateway Chat agent
Chat Slash Commands
-
ee/lib/gitlab/llm/chain/tools/refactor_code/prompts/anthropic.rb
- Refactor -
ee/lib/gitlab/llm/chain/tools/write_tests/prompts/anthropic.rb
- Write tests -
ee/lib/gitlab/llm/chain/tools/explain_code/prompts/anthropic.rb
- Explain Code -
ee/lib/gitlab/llm/chain/tools/explain_vulnerability/executor.rb
- Explain Vulnerability
Experimental Tools
- Summarize Comments Chat
- Fill MR Description
- etc... https://docs.gitlab.com/ee/user/gitlab_duo/#experimental-features
In the prior migration step, we had to support Anthropic's new Message API versus the former Text Completions to migrate from Claude 2 -> 3 models. As long as we don't need to modify our formatting messages, we shouldn't need to do anything within the AI Gateway and only update the monolith. The following is the MR doing that work in the AI Gateway -Support Claude 3 models for Duo Chat.
**This change will be deployed behind a feature flag. **
Links
- Convert documentation tool for Claude 3 (#458811 - closed) • Gosia Ksionek • 17.1
- Migrate Duo Chat tools to use Claude 3 (#455112 - closed) • Nathan Weinshenker • 17.1
- Centralize Instant Model Setting and Consider S... (#444664 - closed) • Jeff Park • 17.0
- Update Code Tasks (Slash Commands) to use Claude 3 (#455717 - closed) • Allen Cook • 17.0 • On track