Prevent duplicate pipeline IIDs -- Last step: Remove workaround behind FF single_pipeline_for_resolver

Background

We currently have the unique index (project_id, iid, partition_id) on the table p_ci_pipelines. This ensures uniqueness within the same partition but not across partitions. Due to a possible workflow bug/race condition, we occasionally save duplicate pipeline iids for the same project, which causes problems such as https://gitlab.com/gitlab-com/request-for-help/-/issues/2855#note_2518831537.

We've explored a couple different possible solutions (https://gitlab.com/gitlab-org/gitlab/-/issues/545167#note_2802709422, https://gitlab.com/gitlab-org/gitlab/-/issues/545167#note_2844491364), and we ultimately decided that the only way to guarantee Pipeline iid uniqueness on the database level is to create a new table to track the iids. (See proposed approach in !210789 (comment 2869213142).)

  • In !213065 (merged), we introduced the static hash partitioned table p_ci_pipeline_iids. It has 64 partitions with the hash on project_id and primary key on (project_id, iid).
  • In !213457 (merged), we added database triggers on p_ci_pipelines so that it keep track of iids in p_ci_pipeline_iids.
  • In !213992 (merged), we are updating the internal ID initial_value logic to read maximum(:iid) from ci_pipeline_iids instead of ci_pipelines.

Implementation

This issue is intended to track the last task in our implementation steps.

Proposal

Following the completion and finalization of #582338 (closed) and #582339 (closed), we can now guarantee iid uniqueness going forward.

Now there should only ever be one pipeline record per iid (in the project scope.) As such, we can remove the workaround (!192174 (merged)) that was implemented behind the FF single_pipeline_for_resolver.