Hold merge requests to a higher standard when master is red
Master has been red for at least 24 hours now, and this is the second month in a row we've had this problem.
Even though we can merge MRs with failing pipelines into master, some engineers are employing a "workaround" where they choose to target their branch against the last green commit on master. This gets them a nice green pipeline on the MR, and makes it more tempting for the maintainer to merge it.
The further behind master a merge request is, the more likely it is that we'll introduce new failures. Indeed, we have had a large number of new failures hit master today, keeping us from getting it to go green.
In this MR, I introduce a new duty on maintainers, to apply a higher standard to MRs when master is red. This happens to invalidate the "last green" strategy, and it will reduce velocity when master is red, but I think we're seeing enough short-term pain, both this month and last, that we need to do something to help this situation.
I don't want to call out people who had been doing the "last green" thing - it's a perfectly reasonable response to master being red for long periods, and the negative consequences of doing so aren't immediately obvious. However, I think it's contributing to our problems and we should do something to stop it.
Is this the right thing to do? WDYT @gitlab-org/maintainers/rails-backend @gitlab-org/frontend etc? Particularly @marin @lbennett @arthanzel , who have all been active in
#development today and talking about the extended-red situation.
Obviously, this is preventation by policy. We have open issues to make it something the tool could enforce automatically: https://gitlab.com/gitlab-org/gitlab-ce/issues/57581 https://gitlab.com/gitlab-org/gitlab-ee/issues/9597