2026-05-12: Sidekiq queueing apdex SLO violation on shard catchall-b (apdex at 70.19%)

Sidekiq queueing apdex SLO violation on shard catchall-b (apdex at 70.19%) (Severity 2)

Problem: Sidekiq jobs on shard catchall-b are experiencing elevated queueing times, dropping queueing apdex to 70.19% and violating SLOs.

Impact: Multiple teams are affected due to Sidekiq job delays on shard catchall-b, with only 70.19% of jobs meeting their queueing targets.

Causes: A backlog of AuditEvents::AuditEventStreamingWorker jobs caused increased queue times. The underlying contributor is a pgbouncer issue, with high connection wait times and many waiting client connections for the Sidekiq pool since around 12:00 UTC.

Response strategy: We deferred the AuditEvents::AuditEventStreamingWorker to reduce backlog and sent a feature flag update notification to the production channel.

This ticket was created to track INC-10084, by incident.io 🔥