Document how to handle old Elasticsearch migrations and branching logic to deal with them

Problem

From !48583 (merged) we need to include branching logic in our code as well as tests for it that handle the state of an index before and after migrations finish.

This adds extra maintenance and since we have no guarantee a migration will run or even finish for a given index we technically would need to keep this logic forever to avoid breaking search in any instance.

Solution

We should set deadlines for when users should have run migrations by and these may align with major releases. Since there are already some expectations around bumping single major releases at a time it might logically follow that we could delete the logic in the subsequent major version and delete the migration itself in the major version after that just to give extra safety net.

We already document that users need to upgrade to latest minor version before bumping major version so that will help with the below plan:

It is suggested to upgrade to the latest available minor version within your major version before proceeding to the next major version. Doing this will address any backward-incompatible changes or deprecations to help ensure a successful upgrade to the next major release. Identify a supported upgrade path.

Proposal

1. We already merged some docs at !54656 (merged) to tell users to check for pending migrations before they do an upgrade
2. We should note clearly in our docs that all migrations must be finished before doing a major version upgrade. We should note this in our ES integration docs as well as make the point stronger in https://docs.gitlab.com/ee/update/#checking-for-pending-elasticsearch-migrations and additionally noting it in https://docs.gitlab.com/ee/update/#upgrading-to-a-new-major-version
3. We make sure it is clear in our docs that you can always fix any broken migrations by recreating the index from scratch with rake gitlab:elastic:index
4. We make it clear that migrations will be removed in major version bumps
5. Create an issue scheduled for %14.0 that lists all the migrations we plan to delete with the following steps documented:
   1. Replace all old migration (basically all except maybe ones we've just created in the minor version before the major release) code with an exception logged to elasticsearch.log that says "This migration has been removed as it needed to run before the major version upgrade. See our docs for how to correct skipped migrations "
   2. The above can be a generic helper where a migration is just marked deleted_migration! and all the internal code removed
   3. Remove the test for migration now that the migration code is gone too
   4. Remove any other code that had to check if the migration was complete and just assume it is complete
6. Update developer documentation for ES migrations with all of the above details
7. Communicate this procedure to self managed support so they are aware of all this and can be sure to advise their clients before doing a major upgrade or easily detect if they might have had a problem caused by doing a major upgrade before finishing a migration