2026-02-05: Sidekiq jobs on low-urgency-cpu-bound shard exceeding queueing duration SLO

Sidekiq jobs on low-urgency-cpu-bound shard exceeding queueing duration SLO (Severity 3)

Problem: A large backlog of Sidekiq jobs built up on the low-urgency-cpu-bound shard, causing queueing performance to drop below the required SLO.

Impact: A sharp increase in queued Sidekiq jobs led to queueing times exceeding the SLO for about 30 minutes, with queue length peaking at over 47,000 jobs. During this period, many jobs did not meet their target queueing time.

Causes: CPU saturation on the low-urgency-cpu-bound shard prevented Sidekiq jobs from being processed quickly enough.

Response strategy: The queue began to drain as CPU saturation was relieved, and by 13:00 Apdex recovered to near-100%. The Sidekiq queueing SLO alert has now resolved.

This ticket was created to track INC-7141, by incident.io 🔥