Postgres transactions timing out; sidekiq queues below apdex score; and overdue pull mirror jobs

Summary

Postgres transactions began timing out

The sidekiq service (main stage) has a apdex score (latency) below SLO

https://dashboards.gitlab.net/d/sidekiq-main/sidekiq-overview?orgId=1

Screen_Shot_2019-12-03_at_12.16.00_PM

More information will be added as we investigate the issue.

Timeline

All times UTC.

2019-12-03

  • 18:04 UTC - Postgres transactions began timing out
  • 18:09 UTC - The sidekiq service (main stage) has a apdex score (latency) below SLO
  • 18:16 UTC - Large number of overdue pull mirror jobs: 10818