Prevent duplicate pipeline IIDs -- Last step: Remove workaround behind FF single_pipeline_for_resolver
Background
We currently have the unique index (project_id, iid, partition_id) on the table p_ci_pipelines. This ensures uniqueness within the same partition but not across partitions. Due to a possible workflow bug/race condition, we occasionally save duplicate pipeline iids for the same project, which causes problems such as https://gitlab.com/gitlab-com/request-for-help/-/issues/2855#note_2518831537.
We've explored a couple different possible solutions (https://gitlab.com/gitlab-org/gitlab/-/issues/545167#note_2802709422, https://gitlab.com/gitlab-org/gitlab/-/issues/545167#note_2844491364), and we ultimately decided that the only way to guarantee Pipeline iid uniqueness on the database level is to create a new table to track the iids. (See proposed approach in !210789 (comment 2869213142).)
- In !213065 (merged), we introduced the static hash partitioned table
p_ci_pipeline_iids. It has 64 partitions with the hash onproject_idand primary key on(project_id, iid). - In !213457 (merged), we added database triggers on
p_ci_pipelinesso that it keep track of iids inp_ci_pipeline_iids. - In !213992 (merged), we are updating the internal ID
initial_valuelogic to readmaximum(:iid)fromci_pipeline_iidsinstead ofci_pipelines.
Implementation
This issue is intended to track the last task in our implementation steps.
Proposal
Following the completion and finalization of #582338 (closed) and #582339 (closed), we can now guarantee iid uniqueness going forward.
Now there should only ever be one pipeline record per iid (in the project scope.) As such, we can remove the workaround (!192174 (merged)) that was implemented behind the FF single_pipeline_for_resolver.