Use Claude 3 Sonnet for Duo Chat Zero Shot

What does this MR do and why?

• This is behind a feature flag for testing
• Uses request body format needed for messages API, which is now supported by the AI Gateway: gitlab-org/modelops/applied-ml/code-suggestions/ai-assist!668 (merged)
• #444629 (closed)

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before	After

How to set up and validate locally

1. In rails console enable the feature flag fully

   Feature.enable(:ai_claude_3_sonnet)

2. Set up the AI Gateway locally
3. Ask Duo Chat questions in your GDK

ee/lib/gitlab/llm/chain/agents/zero_shot/prompts/anthropic.rb

@@ -12,12 +12,9 @@ class Anthropic < Base
 
               def self.prompt(options)
                 if Feature.enabled?(:ai_claude_3_sonnet, options.fetch(:current_user))
-                  prompt = [
-                    Utils::Prompt.default_system_prompt_json
-                    # Utils::Prompt.as_user_json(base_prompt(options))
-                  ] + ::Gitlab::Json.parse(base_prompt(options))
+                  history = truncated_conversation_list(options[:conversation])
 
-                  Requests::Anthropic.prompt(prompt)
+                  text = history + base_prompt(options)
                 else
                   human_role = ROLE_NAMES[Llm::AiMessage::ROLE_USER]
 
@@ -25,28 +22,23 @@ def self.prompt(options)
                   \n\n#{human_role}: #{base_prompt(options)}
                   PROMPT
 
-                  history = truncated_conversation(options[:conversation], Requests::Anthropic::PROMPT_SIZE - text.size)
-                  text = [history, text].join if history.present?
+                  history = truncated_conversation(
+                    options[:conversation],
+                    Requests::Anthropic::PROMPT_SIZE - text.size
+                  )
 
-                  Requests::Anthropic.prompt(text)
+                  text = [history, text].join if history.present?
                 end
+
+                Requests::Anthropic.prompt(text)
               end
               traceable :prompt, name: 'Build prompt', run_type: 'prompt', class_method: true
 
-              def self.truncated_conversation_list(_conversaton, limit)
+              def self.truncated_conversation_list(conversation)
                 return [] if conversation.blank?
 
-                conversation.reverse_each.map do |result, message|
-                  role = ROLE_NAMES[message.role]
-                  buffer = { role: role, content: message.content }
-
-                  break result if buffer.size + result.size > limit
-
-                  next result unless message.role == Llm::AiMessage::ROLE_USER
-
-                  new_str = "#{buffer}#{result}" # Reset the buffer for the next conversation block
-
-                  new_str
+                conversation.map do |message, _|
+                  { role: message.role.to_sym, content: message.content }
                 end
               end