Reinstate the merge request Create pipeline creation requests at the start of PostReceive (FF)

The MR gitlab-org/gitlab!232296 (merged) was reverted to fix the incident production#22243 (closed). This is the follow up issue to reinstate the reverted MR with the additional gate for the entire third argument behind the flag track_ref_pipeline_creation

This ticket was created from INC-10762 using incident.io 🔥


Reinstatement plan

Reinstating gitlab-org/gitlab!232296 (merged) as three stacked MRs, with the third NewMergeRequestWorker argument gated behind track_ref_pipeline_creation.

  • Reconstructed the reverted change on current master
  • Resolved the drift
  • Split into three stacked branches
  • All affected specs green locally
  • Push the three stacked MRs as drafts: gitlab-org/gitlab!256819 (merged), gitlab-org/gitlab!256820 (merged), MR 256821 (replaced by gitlab-org/gitlab!258481 (merged))
  • Update the flag yml metadata (milestone 19.5, introduced_by_url pointing to MR 2)
  • Review and merge MR 1
  • Confirm MR 1 is deployed to 100% of the fleet
  • Merge MR 2 (flag stays off)
  • Merge MR 3 (flag stays off)
  • Begin the flag rollout per rollout issue #597220 (progressive enable, monitor)
  • Verify the auto-merge race is closed for both branch and MR pipelines
  • Full rollout and flag cleanup

Review follow-ups (round 2)

  • Reword the MR 1 commit subject to 72 characters so danger passes
  • Reorder the stack: gitlab-org/gitlab!258481 (merged) (mergeability, replaces MR 256821, ref reader methods, flag) before gitlab-org/gitlab!256820 (merged) (writer)
  • Complete successful non-deferred requests only in UpdateHeadPipelineWorker, after the head pipeline updates (flag on)
  • Don't fail the push when Redis errors while creating ref requests
  • Only create ref requests for branches under git_push_pipeline_limit
  • Review cleanups on gitlab-org/gitlab!256820 (merged): drop safe_failed, keep master's inline completion, keep worker_params unchanged with the flag off, remove the extra comment
  • Add the Redis cost estimate to the writer MR description
  • Push with a full pipeline run and reply to the review threads
Edited by Peter Gumeson