Cancel pipelines when pipeline execution schedule policy is disabled

What does this MR do and why?

When a pipeline execution schedule policy is disabled or deleted, running pipelines created by that policy should be cancelled to free up CI/CD resources and respect the user's intent to stop execution.

Problem

When a scheduled pipeline execution policy is disabled, there may be running, pending, or scheduled pipelines across multiple projects. Users need a way to clean up these pipelines to:

• Free up CI/CD resources immediately
• Prevent unnecessary executions
• Manage costs effectively
• Ensure disabled policies don't continue executing

Solution

This MR implements automatic pipeline cancellation when a pipeline execution schedule policy is unlinked from a project (which happens when the policy is disabled or deleted).

New components:

Component	Description
CancelPolicyPipelinesService	Finds and cancels running pipelines for a given policy, using the security policy bot as the actor
CancelPolicyPipelinesWorker	Async worker triggered during policy sync
Feature flag	cancel_pipelines_when_policy_disabled for safe rollout

Cancellation is triggered when:

• A policy is disabled (enabled: false) → via BaseProjectPolicyService#unlink_policy
• A policy is deleted → via BaseProjectPolicyService#unlink_policy
• A project is removed from policy scope → via BaseProjectPolicyService#unlink_policy

Future work (not in this MR)

• Cancel pipelines when a policy schedule is snoozed (will be added in a follow-up MR)

Implementation details

• Uses the existing security_policy_schedule_pipelines join table (added in !229986 (merged)) to identify which pipelines belong to each policy
• Queries the most recent 100 pipeline records first, then filters by cancelable status for efficiency
• Cancellation is performed by the security policy bot user
• Gated behind a feature flag for safe rollout

Data flow

sequenceDiagram
    participant User
    participant PolicyYAML
    participant SyncPolicyWorker
    participant SyncProjectService
    participant BaseProjectPolicyService
    participant CancelWorker as CancelPolicyPipelinesWorker
    participant CancelService as CancelPolicyPipelinesService
    participant Pipeline as Ci::Pipeline

    User->>PolicyYAML: Disable or delete policy
    PolicyYAML->>SyncPolicyWorker: Policy updated event
    SyncPolicyWorker->>SyncProjectService: Sync for each project
    SyncProjectService->>BaseProjectPolicyService: unlink_policy()
    BaseProjectPolicyService->>CancelWorker: perform_async(policy_id, project_id)
    CancelWorker->>CancelService: execute()
    CancelService->>Pipeline: Find cancelable pipelines via join table
    CancelService->>Pipeline: Cancel each with security bot

Database Review

Queries

Query 1 - Get recent pipeline IDs from security_policy_schedule_pipelines (on gitlab-production-main):

SELECT "security_policy_schedule_pipelines"."pipeline_id" 
FROM "security_policy_schedule_pipelines" 
WHERE "security_policy_schedule_pipelines"."security_policy_id" = 1216926
  AND "security_policy_schedule_pipelines"."project_id" = 74105388
ORDER BY "security_policy_schedule_pipelines"."id" DESC
LIMIT 100;

Query plan New query plan after new index was added: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/52018/commands/153235

Query 2 - Load pipelines by ID and filter by cancelable status (on gitlab-production-ci):

SELECT "p_ci_pipelines".* 
FROM "p_ci_pipelines" 
WHERE "p_ci_pipelines"."id" IN (2542890746, 2542890734, ...)
  AND "p_ci_pipelines"."status" IN ('created', 'waiting_for_resource', 'preparing', 'pending', 'running', 'scheduled');

Query plan

Performance Notes

• The limit of 100 is based on schema constraints: daily schedule (most frequent) + 1 month max time window = ~31 pipelines max. Production data confirms max is 54 pipelines per policy+project in 30 days.
• Queries use existing indexes on security_policy_schedule_pipelines (security_policy_id, project_id, id)
• Pipeline lookups use primary key index on p_ci_pipelines
• Runs asynchronously in a low-urgency Sidekiq worker, only triggered when a policy is disabled (rare event)

Feature Flag

• cancel_pipelines_when_policy_disabled - disabled by default

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

Step 1: Create a group with a security policy project

1. Create a group
2. Create a project with name `security-policies' on the group
3. Add a policy-ci.yml file to the project:

   wait_job:
     script:
       - sleep 60

4. Add a security-policies/.gitlab/security-policies/policy.yml file to the project (replace GROUP_PATH with the path to your group):

   ---
   experiments:
     pipeline_execution_schedule_policy:
       enabled: true
   pipeline_execution_schedule_policy:
   - name: Test Schedule Policy
     description: Policy for testing cancellation
     enabled: false
     content:
       include:
       - project: GROUP_PATH/security-policies
         file: policy-ci.yml
     schedules:
     - type: daily
       start_time: '10:00'
       time_window:
         value: 600
         distribution: random
     policy_scope:
       projects:
         including:
         - id: 93

5. On the group page go to Secure -> Policies and link the project as security policy project
6. Create another project target-project

Step 2: Enable the feature flag

Feature.enable(:cancel_pipelines_when_policy_disabled)

Step 3: Prepare an MR to disable the policy

1. On the group Secure -> Policies select Test Schedule Policy and Edit policy
2. Select Disabled and Update via merge request
3. Don't merge the MR yet

Step 3: Test the cancellation

1. Trigger a scheduled pipeline (replace GROUP_PATH with the path to your group)

   target_project = Project.find_by_full_path('GROUP_PATH/target-project')
   schedule = Security::PipelineExecutionProjectSchedule.find_by(project_id: target_project.id)
   Security::PipelineExecutionPolicies::RunScheduleWorker.new.perform(schedule.id)

2. Go to the target project and select Build -> Pipelines
3. The pipeline should be created, pending or running
4. Merge the MR that disables the policy
5. The pipeline should be cancelled

Related issues

• Closes #560629 (closed)
• Depends on !229986 (merged) (join table - merged)