Skip to content

mini post mortem

A little postmortem for #1247 (Can't save project. Please try again later):

  • 2015-03-18 05:00 (UTC) v7.9rc1 is released and deployed to gitlab.com
  • 2015-03-18 10:00 (UTC) a user notices the bug and reports it to the community forum
  • 2015-03-18 12:00 (UTC) @axil sees the forum entry, creates an issue and informs the core team
  • 2016-03-18 13:00 (UTC) first attempts at locating the error are made by @haynes and @DouweM
  • 2015-03-18 14:00 (UTC) @sytses wakes up and informs @marin
  • 2015-03-18 14:40 (UTC) @marin starts looking into the issue
  • 2015-03-18 14:48 (UTC) @marin fixes the issue on gitlab.com

In total the issue was active on gitlab.com for ~10 hours
and it took ~5 hours from the creation of the bug report to fixing it.
3 of those 5 hours were spend sleeping by half of the team because of the different time zones.

To ensure that the response time is faster in case something like this happens again, @DouweM now has an emergency mail address he can use to wake up @sytses and @dzaporozhets

It's still being investigated how the bug could pass the gitlab test suite without being noticed.