Allow an installation to opt into decomposed mode

We want decomposed mode to be the default in %16.0

Prior to %16.0 there should be a way for installations to "opt-in" earlier if they wish. This also provides a way to test with other GitLab sites such as ops, or dev.

Requirements:

1. We need some guard-rails put in to really ensure there will be no data loss. (see omnibus-gitlab!5492 (comment 1194069933))

   Would the data be retained or would it get cleaned up such that an administrator could lose data? I would like to be sure of the full repercussions to the GitLab administrator and how to fix them so that Support has clear direction on what to do if someone misses the not-ready-for-production call out and panic calls for help.

   1. Avoid split-brain scenario
   2. Allow for migration from decomposed back to single ?
   3. Other scenarios ?

2. Note GitLab.com and staging.gitlab.com are already decomposed

Proposal

TBC