Clean up after incident 8204 (2023-01-05)
During gitlab-com/gl-infra/production#8204 (closed) we created records with mixed partition ids that will need to be fixed for the partitioning work to resume:
gitlabhq_dblab=# select count(*) from p_ci_builds_metadata where partition_id = 101;
 count
--------
 188112
(1 row)
gitlabhq_dblab=# select count(*) from p_ci_builds_metadata inner join ci_builds on ci_builds.id = p_ci_builds_metadata.build_id where p_ci_builds_metadata.partition_id = 101 and ci_builds.partition_id <> 101;
 count
-------
 82282
(1 row)Because of this mismatch we won't be able to update the foreign key between p_ci_builds_metadata and ci_builds to use partition_id. This is likely happening for all 17 tables, but we need to add indexes on partition_id to know more.
Proposal
Add temporary indexes to the following tables and write migrations to update partition_id to 100
| table name | temporary index created | merge requests for indexes | cleanup MR | cleanup completed | 
|---|---|---|---|---|
| ci_build_needs | !108823 (merged) | !111440 (closed) | ||
| ci_build_pending_states | Not needed, will be updated by FK constraint on ci_builds | |||
| ci_build_report_results | !108823 (merged) | !111548 (merged) | ||
| ci_build_trace_chunks | Not needed, will be updated by FK constraint on ci_builds | Not needed | ||
| ci_build_trace_metadata | !109084 (merged) | !111549 (merged) | ||
| ci_builds | !109085 (merged) | !111287 (merged) | ||
| ci_job_artifacts | !109084 (merged) | !111436 (merged) | ||
| ci_job_variables | !109084 (merged) | !111173 (merged) | ||
| ci_pipeline_variables | !108823 (merged) | !111420 (merged) | ||
| ci_pipelines | !109085 (merged) | !109981 (merged) | ||
| ci_sources_pipelines | !109094 (merged) | !109364 (merged) | ||
| ci_stages | !109085 (merged) | !111439 (merged) | ||
| ci_unit_test_failures | Not needed, will be updated by FK constraint on ci_builds | Not needed | ||
| p_ci_builds_metadata | Not needed, already covered by the partitioning constraint | covered by the partitioning constraint | !111576 (merged) | |
| ci_running_builds | !108788 (merged) | Not needed | ||
| ci_pending_builds | !108788 (merged) | Not needed | ||
| ci_builds_runner_session | !108788 (merged) | Not needed, with !109316 (merged) it can be cascaded from ci_builds | 
Edited  by Marius Bobin