Make a good mechanism to revert database changes

This is one of several issues that was raised in https://gitlab.com/gitlab-com/infrastructure/issues/1944#note_38240450 and is required to unblock https://gitlab.com/gitlab-org/gitlab-ce/issues/34877

In "Make sure database migrations are truly backwards compatible (https://gitlab.com/gitlab-org/gitlab-ce/issues/36911)" the work is around preventing needing this issue around reversions; but we should plan for the inevitable as well and make sure reversions are easy.

@stanhu noted: "let's say master introduces 10 new database migrations. Right now to roll them back, we'd have to identify which ones were introduced/breaking the current deploy, and then manually roll them back. We could create some tooling around this to make it less painful." (in https://gitlab.com/gitlab-com/infrastructure/issues/1944#note_38240450)
@smcgivern noted: "This also means that we can't remove a column in a single MR: we need to ignore it, merge that, and then create a separate MR to actually remove it. (As obviously we can't roll back the data otherwise.) How long do we need to leave between those two steps? They can happen in a single release, but not in a single nightly deploy. Is a day enough of a gap?"

@yorickpeterse @_stark , your thoughts?