Support Claude 3 models for Duo Chat
What does this merge request do and why?
Update Duo Chat to use Anthropic Claude 3 Sonne... (gitlab-org/gitlab#444629 - closed)
Claude's new models require usage of the messages API, documented here: https://docs.anthropic.com/claude/reference/messages_post
We do not support the Messages API in the Gateway yet, related issue: Migrate to Anthropic Messages API (#419 - closed)
This MR adds support for the Messages API only when the new models are requested via a header. I tested it locally via http://localhost:5052/docs#/chat/chat_v1_chat_agent_post with "claude-3-opus-20240229"
and "claude-3-sonnet-20240229"
and "claude-3-haiku-20240307"
I did not add logic to support Claude 3 for Code Suggestions. There's a different issue for that: Upgrade to Claude 3 for Code Generation (gitlab-org/gitlab#444405 - closed)
How to set up and validate locally
Ensure AI Gateway is running locally and then run the following curl
request:
curl -X 'POST' \
'http://localhost:5052/v1/chat/agent' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"prompt_components": [
{
"type": "string",
"metadata": {
"source": "string",
"version": "string"
},
"payload": {
"content": [{ "role": "system", "content": "You are a Python engineer"}, { "role": "user", "content": "define a function that adds numbers together" }],
"provider": "anthropic",
"model": "claude-3-sonnet-20240229",
"params": {
"stop_sequences": [
"\n\nHuman",
"Observation:"
],
"temperature": 0.2,
"max_tokens_to_sample": 1024
}
}
}
],
"stream": false
}'
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.