## Overview This epic implements the **Scheduled Pipeline Execution Policy (SPEP) Test Run** feature, which allows users to validate policy configurations and measure pipeline execution duration before enabling policies at scale. ## Problem Statement Customers are hesitant to enable Scheduled Pipeline Execution Policies in production due to lack of visibility into the potential impact. Key concerns include: - **How many CI pipelines will be triggered at once?** - Users need to understand the scale of execution - **How long will it take for all executions to complete?** - Estimation of total execution time is critical - **What is the resource impact?** - Understanding pipeline duration helps estimate infrastructure load For example, a user setting a policy to run on Mondays at 9 PM for 3 hours needs to know: - How many pipelines will trigger per hour? - What is the minimum duration for each pipeline? - Total estimated execution time across all projects Without this information, the feature feels like a "black box," reducing confidence in enabling it in production. ## Solution Implement a **test-run feature** that allows users to: 1. **Save a policy as a draft** (enabled: false in YAML) 2. **Trigger a dry-run** on a single project to estimate impact 3. **View results** showing: - Number of projects in scope - Estimated pipeline duration (from test run) - Estimated total pipelines per hour - Total estimated execution time ### Workflow ``` Save policy as draft ↓ Trigger dry-run on security policy project ↓ Wait for pipeline to complete ↓ Calculate: total_duration × number_of_projects_in_scope ↓ Display results in policy editor ↓ User can enable policy with confidence ``` ### Key Features - **Single test-run per policy** - Only one active test run at a time (not continuous) - **Accurate scope calculation** - Respects policy scope including: - Excluded projects - Included projects - Compliance framework filters - **Real-time status updates** - GraphQL subscription to track test run progress - **Feature-flagged** - Controlled rollout via `spep_test_run` feature flag ## Implementation Phases ### Phase 1: Core Test Run Infrastructure (This Epic) 1. **Feature Flag** - Create `spep_test_run` flag for controlled rollout 2. **Data Model** - `Security::PolicyScheduleTestRun` model and database table 3. **Service Layer** - `TestRunService` to validate and execute test runs 4. **Event Handling** - Subscribe to pipeline completion events to update test run state 5. **GraphQL API** - Mutation to trigger test runs, query to fetch results 6. **Policy Integration** - Add test runs field to security policy type 7. **Real-time Updates** (Optional) - GraphQL subscription for live status 8. **Async Execution** (Optional) - Move pipeline creation to background job 9. **Cleanup** (Optional) - Remove stale test runs when policies change ### Phase 2: UI/UX (Future) - Display test run results in policy editor - Show "Preview" state for policies with completed test runs - Real-time status indicator during test run execution - Results display with estimated impact metrics ## Technical Details ### Database Schema ```sql security_policy_schedule_test_runs: - security_policy_id (FK to security_policies) - project_id (FK to projects) - user_id (FK to users) - pipeline_id (FK to ci_pipelines, optional) - state (enum: running, complete, failed) - error_message (text, max 1000 chars) - created_at, updated_at ``` ### GraphQL Mutation ```graphql mutation { pipelineExecutionSchedulePolicyTestRun(input: { projectPath: "namespace/project" policyName: "my-spep-policy" policyProjectPath: "namespace/security-policies" }) { testRun { id state durationSeconds errorMessage pipeline { id status } } pipeline { id status } errors } } ``` ### GraphQL Query ```graphql query { pipelineExecutionSchedulePolicyTestRun(id: "gid://gitlab/Security::PolicyScheduleTestRun/1") { id state startedAt finishedAt durationSeconds completed errorMessage project { fullPath } pipeline { id status duration } } } ``` ## Acceptance Criteria - [ ] Users can trigger a test run on a policy without enabling it - [ ] Test run executes a pipeline on the security policy project - [ ] Pipeline duration is captured and available via GraphQL - [ ] Test run state updates are reflected in real-time (via subscription or polling) - [ ] Results can be used to estimate impact before enabling the policy - [ ] Feature is behind a feature flag for controlled rollout - [ ] Comprehensive test coverage for all components ## Related Issues - Spike: https://gitlab.com/gitlab-org/gitlab/-/issues/556127 - Parent Epic: https://gitlab.com/groups/gitlab-org/-/work_items/17875 (Scheduled Pipeline Execution Policies - GA Release) ## References - MR: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/218585 (PoC Implementation) - Customer Feedback: Hesitation to enable SPEP without understanding impact on infrastructure