Cancel or re-queue an older pipeline taken from the cache

What does this MR do and why?

A newer pipeline takes the older pipelines of its ref out of Gitlab::Ci::RedundantPipelines::CandidateCache. This MR adds Ci::RedundantPipelines::CancelReplacedPipelineService, which handles one of those older pipelines.

graph LR
    C["CandidateCache<br/>take"] --> S["CancelReplacedPipelineService"]
    S -->|"replaced"| X["canceled<br/>PipelineFamily#cancel"]
    S -->|"may be replaced later"| K["put back<br/>CandidateCache#register"]
    S -->|"gone or finished"| F["dropped"]

Taking a pipeline removes it from the cache. That leaves three outcomes:

• Canceled: the newer pipeline replaces the older one, so PipelineFamily#cancel cancels its family.
• Put back: a later pipeline may still replace it, so the service registers it again.
• Dropped: the pipeline is gone or has finished, so it stays out of the cache.

"Put back" matters when a pipeline runs for the commit the ref already points at. That pipeline cancels nothing. The old search path skipped such pipelines but did not drop them. Without "put back", a second pipeline for the same commit would make the earlier pipelines unreachable.

A later pipeline can be for an earlier commit, for example after a branch is force pushed back. For this reason, the service never cancels the pipeline for the commit the ref points at now (ref_head_sha).

Gitlab::Ci::RedundantPipelines::AutoCancelPolicy#protected? now reads protection from redis as well as from the interruptible_protected column. Redis protection uses CandidateCache#protected?, added in !257741 (merged). The worker writes the column some time after the job starts. Until then, only redis knows that the pipeline is protected.

The policy reads the column first, because with_auto_cancel_policy_associations already preloads it. It asks redis only when the column says no. That costs at most one ZSCORE per policy, and only in conservative mode. The cache is injected into the policy and defaults to CandidateCache.for(project_id:, ref:).

The service loads the older pipeline by primary key and partition with Ci::Pipeline.find_by_id_and_partition. It uses the existing PipelineFamily queries and adds no new query shapes.

Nothing calls CancelReplacedPipelineService yet. The caller, CancelSupersededPipelinesService, arrives in a later merge request.

Everything is behind the feature flag ci_redundant_pipeline_candidates_cache, type wip, disabled by default.

How to set up and validate locally

bundle exec rspec spec/services/ci/redundant_pipelines/cancel_replaced_pipeline_service_spec.rb spec/lib/gitlab/ci/redundant_pipelines/auto_cancel_policy_spec.rb

References

• Discussion issue: #621543
• Parent proof-of-concept MR: !253751
• Depends on: !257741 (merged)

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.