GitLab.com outage Thu 17 Aug ~15:37 UTC

/cc @gl-infra

Timeline of events (UTC):

  • 15:37 Load on primary DB increases and we start returning 5xx errors
  • 15:37 32K IOPS spike
  • 15:42 GitLab.com is back
  • 15:45 Another spike on primary database ~14K IOPS
  • 15:50 GitLab.com unavailable
  • 15:53 Another spike on primary database ~40K IOPS
  • 15:56 GitLab.com is back
  • 16:01 Another spike on primary database ~36K IOPS
  • 16:02 GitLab.com unavailable
  • 16:04 Another spike on primary database ~61K IOPS
  • 16:07 GitLab.com is back
Edited by Victor Lopez