Skip to content

feat: track and validate feature usage for multi purpose endpoint

Shinya Maeda requested to merge feature-category-tracking-for-multi-purpose-endpoints into main

NOTE: This MR has high-priority due to the timeline.

What does this merge request do and why?

This MR implements feature usage tracking and validation for multi-purpose endpoints, which is based on the AI Gateway ADR 002: Exposing proxy endpoints to AI providers.

The main goal of these endpoints is to enable the independent AI features in self-managed instances within the proposed timeline. See the issue and this issue for more information.

How to set up and validate locally

  1. poetry run ai_gateway. Enable AIGW_AUTH__BYPASS_EXTERNAL for testing purpose.
  2. Execute the following curl CLI in your terminal:

Request:

curl -X 'POST' \
  'http://0.0.0.0:5052/v1/proxy/anthropic/v1/complete' \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -H 'X-Gitlab-Unit-Primitive: analyze_ci_job_failure' \
  --header "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-2.1",
    "max_tokens_to_sample": 1024,
    "prompt": "\n\nHuman: Hello, Claude\n\nAssistant:"
}'

Log:

{
    "url": "http://0.0.0.0:5052/v1/proxy/anthropic/v1/complete",
    "path": "/v1/proxy/anthropic/v1/complete",
    "status_code": 200,
    "method": "POST",
    "correlation_id": "07fed5a21d8f4e2e89d70ce42b7fdacc",
    "http_version": "1.1",
    "client_ip": "127.0.0.1",
    "client_port": 35610,
    "duration_s": 0.9250697800016496,
    "duration_request": -1,
    "cpu_s": 0.013949134999999835,
    "user_agent": "curl/7.81.0",
    "gitlab_instance_id": null,
    "gitlab_global_user_id": null,
    "gitlab_host_name": null,
    "gitlab_saas_duo_pro_namespace_ids": null,
    "gitlab_saas_namespace_ids": null,
    "gitlab_realm": null,
    "meta.feature_category": "continuous_integration",
    "meta.unit_primitive": "analyze_ci_job_failure",
    "logger": "api.access",
    "level": "info",
    "type": "mlops",
    "stage": "main",
    "timestamp": "2024-05-13T09:37:31.284205Z",
    "message": "127.0.0.1:35610 - \"POST /v1/proxy/anthropic/v1/complete HTTP/1.1\" 200"
}

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Shinya Maeda

Merge request reports