Create a list of scenarios for 'wheel of misfortune' for incident practice
In order to start practicing incident response on some level, it would be good to have a seed set of incident scenarios.
Suggested location - new folder in https://gitlab.com/gitlab-com/runbooks
Suggested ideas:
- Patroni node failover.
- Loss of (power off) of git file node
- Severe degredation of performance - rollback of deploy?
Open to more ideas...
Each scenario will be a new .md in this folder. For now we can have the doc just describe the scenario. In the future, this seems like it would be a good start for chaos ops ideas too.
cc @gitlab-com/gl-infra @gitlab-com/gl-security for more ideas and feedback.