Create an anthropic proxy endpoint in the AI gateway
What does this merge request do and why?
This MR creates a proxy endpoint that forwards the requests to anthropic. The only change it makes is converting the prompt to a format that anthropic expects (Human: {prompt}\nAssistant: {answer}). It sets some defaults, but all can be overridden with parameters.
How to set up and validate locally
Set ANTHROPIC_API_KEY
ENV to your Anthropic API key. Make a call to anthropic
curl -X 'POST' \
'http://0.0.0.0:5052/v2/anthropic' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <access_token>' \
-H 'X-Gitlab-Authentication-Type: oidc' \
-d '{
"prompt": "your prompt here",
"model": "claude-2",
"max_tokens_to_sample": 2048,
"stop_sequences": [],
"stream": false,
"temperature": 0.1
}'
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Closes #289 (closed)
Edited by Roy Zwambag