Decomposition: Migration plan for moving CI tables on GitLab.com
### Overview As a first step of https://gitlab.com/groups/gitlab-org/-/epics/6168 to decompose our single logical database into multiple logical databases (that will live on separate database clusters on GitLab.com), we are planning to move all `ci_*` tables to a new database. The family of `ci_*` tables consist of a large percentage (\~35%) of our current Database size, so this migration will be one of the largest ones, if not the largest one, that we have to perform. That makes the logistics and process we have to follow on GitLab.com and large self-managed instances pretty unique and different than the approach that we may have to take for smaller instances, especially ones where the shards will live as logical databases under the same Database Server. For that reason, we are going to focus first on how to address the problem for GitLab.com and large self-managed instances that already use a cluster of databases (primary and replicas managed by patroni) and follow-up with solutions for smaller single database instances once we successfully solve the problem on GitLab.com's scale. At the same time, moving such a large percentage of our current Database, enables us to investigate a solution that involves replicating the whole database, switching all `ci_*` related models to use the new shard once we make sure that it is synced and then drop all non CI related tables from that shard (rest \~65% of the database) ### Scope In this epic we are going to finalize our plan to migrate **all** the `ci_*` tables on **GitLab.com**. We are going to test the steps necessary for its success and successfully conclude the epic once we build enough confidence on it working without issues. We are going to continue with the implementation on a follow-up epic and then work on how to address this for smaller single database instances.
epic