Skip to content

Migrate AI features to Claude 3.5 Sonnet

Description

The objective is to migrate from Claude 3 Sonnet to Claude 3.5 Sonnet across all relevant platforms and tools within the organization. This migration aims to leverage the improved performance and capabilities of Claude 3.5 Sonnet, which has demonstrated superior results in initial evaluations.

We successfully migrated tools to Claude 3 models during %17.0 milestone, so the upcoming work should follow a similar path to Migrate Duo Chat tools issue.

Tasks

To scope out the work, we should consider the following AI features based on Unit Primitives that are grouped together. The following Duo Chat features gives us a sense just for Duo Chat

Duo Chat AIGW Agent:

Duo Chat tool:

Chat Tools

  • ee/lib/gitlab/llm/chain/tools/ci_editor_assistant/prompts/anthropic.rb - CI Editor
  • ee/lib/gitlab/llm/chain/tools/gitlab_documentation/executor.rb - Gitlab Documentation
  • ee/lib/gitlab/llm/chain/tools/epic_reader/prompts/anthropic.rb - Epic Reader
  • ee/lib/gitlab/llm/chain/tools/issue_reader/prompts/anthropic.rb - Issue Reader

NOT PUT INTO PRODUCTION YET

Chat Slash Commands

  • ee/lib/gitlab/llm/chain/tools/refactor_code/prompts/anthropic.rb - Refactor
  • ee/lib/gitlab/llm/chain/tools/write_tests/prompts/anthropic.rb - Write tests
  • ee/lib/gitlab/llm/chain/tools/explain_code/prompts/anthropic.rb - Explain Code
  • ee/lib/gitlab/llm/chain/tools/explain_vulnerability/executor.rb - Explain Vulnerability

Experimental Tools

In the prior migration step, we had to support Anthropic's new Message API versus the former Text Completions to migrate from Claude 2 -> 3 models. As long as we don't need to modify our formatting messages, we shouldn't need to do anything within the AI Gateway and only update the monolith. The following is the MR doing that work in the AI Gateway -Support Claude 3 models for Duo Chat.

**This change will be deployed behind a feature flag. **

Links

Edited by Nathan Weinshenker