Skip to content

Add guidance on reverting database migration change in "broken master" process

Nao Hashizume requested to merge nh-add-docs-for-revert-migration into master

Why is this change being made?

We recently had a broken master incident (gitlab-org/gitlab#387761 (closed)) that needed to undo the database migration executed in staging and block the deployment to prevent the migration from running on production. I reverted the MR that introduced the migration, but learned that the revert was insufficient from @tkuah's below comment:

Quoting the comment from @tkuah in gitlab-org/gitlab!108644 (comment 1235023691):

FYI I have opened gitlab-com/gl-infra/production#8229 (closed) because a revert is not sufficient - gitlab-org/gitlab!104155 (merged) might be deployed; and then another deploy that includes this MR. This will then make GitLab.com DB schema inconsistent with everything else

This MR is to update the master-broken documentation to reflect the additional step that must be taken when reverting database migration changes.

Author Checklist

  • Provided a concise title for this Merge Request (MR)
  • Added a description to this MR explaining the reasons for the proposed change, per say why, not just what
    • Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added.
  • Assign reviewers for this MR to the correct Directly Responsible Individual/s (DRI)
    • If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the Maintained by section on the page being edited
    • If your manager does not have merge rights, please ask someone to merge it AFTER it has been approved by your manager in #mr-buddies
  • If the changes affect team members, or warrant an announcement in another way, please consider posting an update in #whats-happening-at-gitlab linking to this MR
    • If this is a change that directly impacts the majority of global team members, it should be a candidate for #company-fyi. Please work with internal communications and check the handbook for examples.

Edited by Nao Hashizume

Merge request reports