feat: anthropic proxy endpoint
NOTE: This MR has high-priority due to the timeline.
What does this merge request do and why?
This MR implements Anthropic proxy endpoint according to the AI Gateway ADR 002: Exposing proxy endpoints to AI providers. The main goal of these endpoints is to enable the independent AI features in self-managed instances within the proposed timeline. See the issue and this issue for more information.
For GitLab-Rails counter-part, see Anthropic client requests via AI Gateway proxy (gitlab-org/gitlab!150255 - closed)
How to set up and validate locally
-
poetry run ai_gateway
. EnableAIGW_AUTH__BYPASS_EXTERNAL
for testing purpose. - Execute the following
curl
CLI in your terminal:
curl -X 'POST' \
'http://0.0.0.0:5052/v1/proxy/anthropic/v1/complete' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
--header "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-2.1",
"max_tokens_to_sample": 1024,
"prompt": "\n\nHuman: Hello, Claude\n\nAssistant:"
}'
curl -X 'POST' \
'http://0.0.0.0:5052/v1/proxy/anthropic/v1/messages' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
--header "anthropic-version: 2023-06-01" \
-d '{
"model": "claude-3-opus-20240229",
"max_tokens": 1024,
"messages": [
{"role": "user", "content": "Can you sign a long song?"}
],
"stream": true
}'
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Shinya Maeda