Skip to content

Update the broken master process to take the Pipeline for Merged Results in account

Rémy Coutable requested to merge merged-results-in-broken-master-process into master

Why is this change being made?

This is a preparation step for gitlab-org/gitlab#200037 (closed).

Summary of the changes:

  1. Change the broken master triage SLOs to 4 hours instead of 12 hours since it's closest to the reality.
  2. If a revert is performed, create an issue to reinstate the merge request and assign it to the author of the reverted merge request.
  3. If the broken master affects any auto-deploy, add the relevant ~"Pick into auto-deploy" label.
  4. If the broken master affects any stable branches (e.g. gitlab-org/gitlab!25274 (merged)), open new merge requests directly against the stable branches which are broken and ping the current release manager in the merge requests to avoid delays in releases / security releases. Optionally, post a message in #releases.
  5. Once the resolution DRI announces that master is fixed:
    • Maintainers should start a new Pipeline for Merged Results (for canonical MRs) and enable "Merge When Pipeline Succeeds" (MWPS).
    • (For forks only) Authors should rebase their open merge requests (since Pipeline for Merged Results isn't supported in these cases).
  6. Most merge requests don't need to be merged immediately, so the priority should always be maintaining the stability of master over keeping a high throughput.
  7. Merging during a broken master event is now only allowed if the master is broken for more than 2 hours.
  8. Whether the merge requests pipeline is failing or not, if master is red:
    • For canonical merge requests, always start a new merge request pipeline and check that the failure happens in master with a link to the ~"master:broken" issue before clicking the red "Merge" button.
    • For forks, check how far behind master the source branch is. If it's more than 100 commits behind master, ask the author to rebase it before merging.
    • This reduces the chance of introducing new failures, and also acts to slow (but not stop) the rate of change in master, helping us to make it green again.

Review app: https://merged-results-in-broken-master-process.about.gitlab-review.app/handbook/engineering/workflow/index.html

Does this MR meet the acceptance criteria?

Conformity

Edited by Rémy Coutable

Merge request reports