Skip to content

Blocked: Start generative AI server with basic authentication

Alexander Chueshev requested to merge basic-auth-generative-api into internal-model-gateway

This MR adds basic authentication to the PALM model API endpoint introduced in !59 (merged). We consider basic authentication as a solution for internal use cases - experiments.

How to setup:

Please, set username and password using env variables:

GENERATIVE_AI_ONLY=True
GENERATIVE_AI_USERNAME="username"
GENERATIVE_AI_PASSWORD="password"

Query the PALM text model using curl:

curl -v -X POST \
  -H "Content-Type: application/json" \
  -H "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=" \
  "http://localhost:5052/v2/generate/palm" \
  --data-binary @- <<EOF
{
  "prompt_version": 1,
  "model": {
    "name": "text-bison-001",
    "content": "string",
    "temperature": 0.2,
    "max_decode_steps": 16,
    "top_p": 0.95,
    "top_k": 40
  }
}
EOF

Ref: #64

Edited by Alexander Chueshev

Merge request reports