Create a list of scenarios for 'wheel of misfortune' for incident practice

In order to start practicing incident response on some level, it would be good to have a seed set of incident scenarios.

Suggested location - new folder in https://gitlab.com/gitlab-com/runbooks

Suggested ideas:

  • Patroni node failover.
  • Loss of (power off) of git file node
  • Severe degredation of performance - rollback of deploy?

Open to more ideas...

Each scenario will be a new .md in this folder. For now we can have the doc just describe the scenario. In the future, this seems like it would be a good start for chaos ops ideas too.

cc @gitlab-com/gl-infra @gitlab-com/gl-security for more ideas and feedback.