Add details on what to do when master is broken: revert or quarantine
- Now that https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26920 is merged, we only run MySQL and PG 10 on
masterand on schedules which means that we may find problems specific to those only once it hits
- It was suggested in https://gitlab.com/gitlab-org/quality/production/issues/4#note_157556285 that we should be more aggressive in reverting MRs that introduce such failures.
- With https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/26934 we stopped automatically retrying jobs that fail at the
scriptstep, meaning that flaky tests can now lead to a broken pipeline, so I think we need to be more aggressive about putting specs in quarantine now since we won't retry those automatically at the job level, which means that we'll detect them more easily but also that
mastermay fail more often because of that (but at least it won't be "hidden" by automatic retries).
This workflow update addresses those 2 things.