Enable schedule policy test runs
What does this MR do and why?
This MR enables the test-run feature for Scheduled Pipeline Execution Policies (SPEP). The feature allows users to validate policy configurations and measure pipeline execution duration before enabling policies at scale.
Problem Statement
Customers are hesitant to enable Scheduled Pipeline Execution Policies due to lack of visibility into the potential impact. They need to understand:
- How many pipelines will be triggered at once
- Estimated execution duration for all pipelines
- Resource utilization impact on their infrastructure
Changes
This MR introduces:
-
TestRunService: Service to trigger test-runs on a selected project with proper authorization and validation -
CreateScheduledPipelineService: Extracted pipeline creation logic (refactored fromRunScheduleWorker) for reuse between scheduled runs and test-runs -
TestRunCompletionWorker: Event subscriber that updates test-run status when the pipeline completes -
Feature Flag:
spep_test_runfor controlled rollout
Key Features
- Single Project Selection: Test on one project at a time
- Developer Permission: Users with developer access to the security policy project can initiate test-runs
- Duration Tracking: Captures pipeline execution duration and success/failure status via the associated pipeline
- Policy Scope Validation: Ensures the project is within the policy's scope
References
- Spike issue: #556127 (closed)
- PoC MR: !218585 (closed)
- Parent Epic: &17875
How to set up and validate locally
Prerequisites
-
Enable the feature flag:
Feature.enable(:spep_test_run) -
Ensure you have:
- A group with at least one project
- A separate project to use as the security policy project
- Developer access to the security policy project
1. Create a Security Policy Project
Create a new project (e.g., your-group/security-policies) that will hold your security policies.
2. Link the Security Policy Project to your Group
Go to your group's Settings > Security > Policies and link the security policy project you created. Alternatively, via Rails console:
group = Group.find_by_full_path('your-group')
policy_project = Project.find_by_full_path('your-group/security-policies')
Security::OrchestrationPolicyConfiguration.create!(
namespace: group,
security_policy_management_project: policy_project
)
3. Set up a Scheduled Pipeline Execution Policy
Create a .gitlab/security-policies/policy.yml in your security policy project:
---
experiments:
pipeline_execution_schedule_policy:
enabled: true
pipeline_execution_policy: []
approval_policy: []
pipeline_execution_schedule_policy:
- name: test-policy
description: 'Test scheduled policy'
enabled: true
content:
include:
- project: your-group/security-policies
file: policy-ci.yml
schedules:
- type: daily
start_time: '10:00'
time_window:
value: 600
distribution: random
Create a policy-ci.yml in the same project:
test-job:
stage: test
script:
- echo "Running scheduled policy test"
- sleep 10
4. Trigger a test-run via Rails console
# Find the policy
policy = Security::Policy.find_by(name: 'test-policy')
# Find a project in scope (any project in the group)
project = Project.find_by_full_path('your-group/your-project')
# Get a user with developer access to the security policy project
user = User.find_by_username('your-username')
# Execute the test-run
result = Security::PipelineExecutionPolicies::TestRunService.new(
policy: policy,
project: project,
user: user
).execute
# Check the result
result.success? # => true
result.payload[:test_run] # => Security::ScheduledPipelineExecutionPolicyTestRun
result.payload[:pipeline] # => Ci::Pipeline
5. Verify test-run completion
After the pipeline finishes, verify the test-run status was updated:
test_run = result.payload[:test_run].reload
test_run.state # => "complete" or "failed"
test_run.pipeline.duration # => pipeline duration in seconds
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #586697 (closed)