2026-09-09: Redis memory utilization near max on redis-cluster-shared-state

Redis memory utilization near max on redis-cluster-shared-state (Severity 2)

Problem: Redis memory usage on shard 04 in main rose to the maxmemory limit, putting the redis-cluster-shared-state service at risk of failing write requests.

Impact: Redis on shard 04 reached 100% memory, which put write requests at risk. Customers also reported elevated pending CI jobs, and some job webhooks and integration notifications may have been delayed or lost.

Causes: A large queued-job list for the CI build hooks worker used a huge amount of Redis memory on shard 04. Disabling the concurrency limit and deleting that backlog key relieved the pressure.

Response strategy: We deleted the throttled-job backlog key to reduce Redis load, and memory has started to fall. We also temporarily disabled the Sidekiq concurrency limit for the CI build hooks worker.