2025-09-01: No traffic detected by runway_lb load balancer for ai-gateway

No traffic detected by runway_lb load balancer for ai-gateway (Severity 4 (Low))

Problem: The runway_lb load balancer for the ai-gateway service is experiencing intermittent visibility of HTTP traffic metrics due to potential issues with Google Stackdriver API timeouts.

Impact: No direct impact on customers has been identified as service operations appear healthy.

Causes: The issue is likely caused by Google Stackdriver API timeouts, leading to intermittent disappearance and reappearance of monitoring data.

Response strategy: The Stackdriver API timeout has been increased as a potential solution, the backoff has been tweaked, and we have reduced the window we load from stackdriver in a single window to 2m

---

This ticket was created to track INC-3645, by incident.io 🔥