Skip to content

feat: use multiprocess metrics automatically in Docker image

Stan Hu requested to merge sh-use-multiprocess-metrics-docker into main

What does this merge request do and why?

This is a follow-up merge request to !776 (merged).

If a user sets WEB_CONCURRENCY to a value > 1 so that multiple worker processes are spawned, automatically enable Prometheus multiprocess metrics by setting the PROMETHEUS_MULTIPROC_DIR to a temporary directory. This will enable anyone running the Docker image to scrape the metrics endpoint and get aggregate data from all the workers.

How to set up and validate locally

  1. Build the image: docker build -t ai-gateway-test:latest .
  2. Run the image: docker run -it -p 5052:5052 -p 8082:8082 -e AIGW_FASTAPI__API_PORT=5052 -e AIGW_MOCK_MODEL_RESPONSES=true -e WEB_CONCURRENCY=2 -e AIGW_AUTH__BYPASS_EXTERNAL=true --name ai-gateway-test ai-gateway-test:latest
  3. You should see a startup message in the form: Storing multiprocess metrics in /tmp/ai_gateway.Pd5Rmm...
  4. Make some requests to the AI gateway.
  5. Run curl http://localhost:8082/metrics.

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Stan Hu

Merge request reports