2025-12-02: Error rate SLO violation on git service loadbalancer and workhorse components

Error rate SLO violation on git service loadbalancer and workhorse components (Severity 3)

Problem: The git service experienced elevated 5xx error rates in loadbalancer and workhorse components, violating SLOs.

Impact: Elevated error rates for 5xx responses in both loadbalancer and workhorse components of the git service, with error rates up to 0.201%, exceeded SLOs across multiple environments and regions.

Causes: A Google Cloud host failure and automatic restart of the gitaly-08-stor-gprd node caused a significant drop in service operations and connection errors during the disruption window.

Response strategy: The affected node automatically restarted, and service metrics have since recovered.

---

This ticket was created to track INC-6077, by incident.io 🔥