chore: increase min instances and resources (!489) · Merge requests · GitLab.org / ModelOps / AI Assisted (formerly Applied ML) / Code Suggestions / AI Gateway · GitLab

Chance Feick requested to merge chore/runway-resources into main Dec 08, 2023

What

Increase minimum number of instances, memory and CPU limits for AI gateway. Correlation w/ autoscaling and resource saturation:

Why

Service overview dashboard has degraded apdex and error rate: https://dashboards.gitlab.net/d/ai-gateway-main/ai-gateway3a-overview?orgId=1&from=1701985538464&to=1701992760349. Alerts have triggered in #g_mlops-alerts: https://gitlab.slack.com/archives/C0586SBDZU2/p1701989326049019.

Context: https://gitlab.slack.com/archives/C052QHHFNH0/p1701989741377769.