Enable schedule policy test runs

What does this MR do and why?

This MR enables the test-run feature for Scheduled Pipeline Execution Policies (SPEP). The feature allows users to validate policy configurations and measure pipeline execution duration before enabling policies at scale.

Problem Statement

Customers are hesitant to enable Scheduled Pipeline Execution Policies due to lack of visibility into the potential impact. They need to understand:

  • How many pipelines will be triggered at once
  • Estimated execution duration for all pipelines
  • Resource utilization impact on their infrastructure

Changes

This MR introduces:

  1. TestRunService: Service to trigger test-runs on a selected project with proper authorization and validation
  2. CreateScheduledPipelineService: Extracted pipeline creation logic (refactored from RunScheduleWorker) for reuse between scheduled runs and test-runs
  3. TestRunCompletionWorker: Event subscriber that updates test-run status when the pipeline completes
  4. Feature Flag: spep_test_run for controlled rollout

Key Features

  • Single Project Selection: Test on one project at a time
  • Developer Permission: Users with developer access to the security policy project can initiate test-runs
  • Duration Tracking: Captures pipeline execution duration and success/failure status via the associated pipeline
  • Policy Scope Validation: Ensures the project is within the policy's scope

References

How to set up and validate locally

Prerequisites

  1. Enable the feature flag:

    Feature.enable(:spep_test_run)
  2. Ensure you have:

    • A group with at least one project
    • A separate project to use as the security policy project
    • Developer access to the security policy project

1. Create a Security Policy Project

Create a new project (e.g., your-group/security-policies) that will hold your security policies.

Go to your group's Settings > Security > Policies and link the security policy project you created. Alternatively, via Rails console:

group = Group.find_by_full_path('your-group')
policy_project = Project.find_by_full_path('your-group/security-policies')

Security::OrchestrationPolicyConfiguration.create!(
  namespace: group,
  security_policy_management_project: policy_project
)

3. Set up a Scheduled Pipeline Execution Policy

Create a .gitlab/security-policies/policy.yml in your security policy project:

---
experiments:
  pipeline_execution_schedule_policy:
    enabled: true
pipeline_execution_policy: []
approval_policy: []
pipeline_execution_schedule_policy:
  - name: test-policy
    description: 'Test scheduled policy'
    enabled: true
    content:
      include:
        - project: your-group/security-policies
          file: policy-ci.yml
    schedules:
      - type: daily
        start_time: '10:00'
        time_window:
          value: 600
          distribution: random

Create a policy-ci.yml in the same project:

test-job:
  stage: test
  script:
    - echo "Running scheduled policy test"
    - sleep 10

4. Trigger a test-run via Rails console

# Find the policy
policy = Security::Policy.find_by(name: 'test-policy')

# Find a project in scope (any project in the group)
project = Project.find_by_full_path('your-group/your-project')

# Get a user with developer access to the security policy project
user = User.find_by_username('your-username')

# Execute the test-run
result = Security::PipelineExecutionPolicies::TestRunService.new(
  policy: policy,
  project: project,
  user: user
).execute

# Check the result
result.success? # => true
result.payload[:test_run] # => Security::ScheduledPipelineExecutionPolicyTestRun
result.payload[:pipeline] # => Ci::Pipeline

5. Verify test-run completion

After the pipeline finishes, verify the test-run status was updated:

test_run = result.payload[:test_run].reload
test_run.state # => "complete" or "failed"
test_run.pipeline.duration # => pipeline duration in seconds

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #586697 (closed)

Edited by Andy Schoenen

Merge request reports

Loading