feat: use multiprocess metrics automatically in Docker image
What does this merge request do and why?
This is a follow-up merge request to !776 (merged).
If a user sets WEB_CONCURRENCY
to a value > 1 so that multiple
worker processes are spawned, automatically enable Prometheus
multiprocess metrics by setting the PROMETHEUS_MULTIPROC_DIR
to a
temporary directory. This will enable anyone running the Docker image
to scrape the metrics endpoint and get aggregate data from all the
workers.
How to set up and validate locally
- Build the image:
docker build -t ai-gateway-test:latest .
- Run the image:
docker run -it -p 5052:5052 -p 8082:8082 -e AIGW_FASTAPI__API_PORT=5052 -e AIGW_MOCK_MODEL_RESPONSES=true -e WEB_CONCURRENCY=2 -e AIGW_AUTH__BYPASS_EXTERNAL=true --name ai-gateway-test ai-gateway-test:latest
- You should see a startup message in the form:
Storing multiprocess metrics in /tmp/ai_gateway.Pd5Rmm...
- Make some requests to the AI gateway.
- Run
curl http://localhost:8082/metrics
.
Merge request checklist
-
Tests added for new functionality. If not, please raise an issue to follow up. -
Documentation added/updated, if needed.
Edited by Stan Hu