2026-04-20: The ci_runner_jobs SLI of the ci-runners service on shard `saas-linux-small-amd64` has an apdex violating SLO
# The ci_runner_jobs SLI of the ci-runners service on shard `saas-linux-small-amd64` has an apdex violating SLO (Severity 2)
**Problem**: The apdex score for ci_runner_jobs on the saas-linux-small-amd64 shard of the ci-runners service dropped to 43.59%, violating the SLO. This caused CI pipelines to be delayed and jobs to remain pending.
**Impact**: Multiple users experienced CI pipeline delays and jobs stuck in pending on the saas-linux-small-amd64 shard. This resulted in a significant backlog of jobs and an increase in support ticket volume about job pickup failures. Service has now recovered, with Apdex returning to healthy levels and job queues trending down.
**Causes**: Confirmed capacity issues in the original cloud zone caused CI runner capacity exhaustion on the saas-linux-small-amd64 shard, which prevented new CI jobs from being picked up.
**Response strategy**: We reconfigured runner nodes to use the us-east1-c zone to relieve capacity exhaustion and reduce the job queue backlog. All nodes were updated and reloaded with new configuration. Monitoring confirms that Apdex has recovered and job queues are clearing. A follow-up has been created to expand hosted runner support to a third zone and reduce risk from future zonal issues.
_This ticket was created to track_ [_INC-9343_](https://app.incident.io/gitlab/incidents/9343)_, by_ [_incident.io_](https://app.incident.io) 🔥
issue