Cancel redundant pipeline synchronously during a single job execution

What does this MR do and why?

Cancel redundant pipeline synchronously during a single job execution

This is not an urgent process, it's a nice-to have optimization. If there are a large number of Pipelines to cancel, we should still cancel them iteratively from one worker, instead of queuing a separate job for each Pipeline to be cancelled and potentially flooding the background workers with thousands of jobs that will attempt to execute simultaneously.

References

• Ongoing epic: Improve long-term scalability of CancelRedundan... (&12419 - closed)
• Specific complaint, potentially related: https://gitlab.com/gitlab-com/ops-sub-department/section-ops-request-for-help/-/issues/484

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.