Write post-mortem
We need to write a post-mortem about what happened with the database, how we recovered it, and what we're going to do in the future to prevent this from ever happening again. This particular blog should mention the following issues (all related):
- https://gitlab.com/gitlab-com/infrastructure/issues/1094
- https://gitlab.com/gitlab-com/infrastructure/issues/1095
- https://gitlab.com/gitlab-com/infrastructure/issues/1096
- https://gitlab.com/gitlab-com/infrastructure/issues/1097
- https://gitlab.com/gitlab-com/infrastructure/issues/1098
- https://gitlab.com/gitlab-com/infrastructure/issues/1099
- https://gitlab.com/gitlab-com/infrastructure/issues/1100
- https://gitlab.com/gitlab-com/infrastructure/issues/1101
- https://gitlab.com/gitlab-com/infrastructure/issues/1102
- https://gitlab.com/gitlab-com/infrastructure/issues/1103
- https://gitlab.com/gitlab-com/infrastructure/issues/1104
- https://gitlab.com/gitlab-com/infrastructure/issues/1105
I'll start writing a draft tomorrow.
cc @gl-infra