Add experiment to manage stage inject strategy

What does this MR do and why?

This MR adds an experiment to control how Pipeline Execution Policy stages are injected into project pipelines, addressing a critical limitation where policy stages could not be positioned at the beginning of pipelines.

Problem

Currently, when Pipeline Execution Policies define custom stages, they are injected using a "project stages first" strategy. This means:

• Policy stages are appended after all project-defined stages
• Policy jobs cannot run at the beginning of the pipeline unless explicitly using .pipeline-policy-pre
• Organizations cannot enforce security/compliance checks before project stages when those checks need to run after .pre (e.g., scans that depend on build artifacts)

Example of current behavior:

• Policy stages: [.pipeline-policy-pre, .pre, policy-build, policy-test, policy-deploy, .post]
• Project stages: [build, test, deploy]
• Result: [.pipeline-policy-pre, .pre, build, test, deploy, policy-build, policy-test, policy-deploy, .post]

Solution

This MR introduces a new stage injection strategy controlled by an experiment flag: pipeline_execution_policy_stages_higher_precedence

When enabled, policy stages are given higher precedence by reversing the merge order:

1. First, all policy stages are merged together
2. Then, project stages are merged into the policy stages

This allows policy stages to be positioned at the beginning of the pipeline (after .pre) when they have no common stages with the project, while still respecting stage dependencies through DAG-based topological sorting.

Example with new strategy:

• Policy stages: [.pipeline-policy-pre, .pre, policy-build, policy-test, policy-deploy, .post]
• Project stages: [build, test, deploy]
• Result: [.pipeline-policy-pre, .pre, policy-build, policy-test, policy-deploy, build, test, deploy, .post]

Implementation Details

The MR adds:

1. Two injection strategies in Gitlab::Ci::Config::StagesMerger:

   • :project_stages_first (default, existing behavior) - merges policy stages into project stages
   • :policy_stages_first (new, experimental behavior) - merges project stages into policy stages

2. Experiment configuration in policy configuration:

   experiments:
     pipeline_execution_policy_stages_higher_precedence:
       enabled: true

3. DAG-based dependency resolution using topological sort:

   • When policy stages have no common stages with project stages, they appear first
   • When policy stages share common stages with project stages, dependencies are respected
   • Cyclic dependencies are detected and raise an error
   • Multiple policies are merged in order before being merged with project stages

How it works:

# Strategy: :policy_stages_first
# 1. Merge all policy stages together
merged_policy_stages = merge_and_order(policy1_stages, policy2_stages, ...)

# 2. Merge project stages into the merged policy stages
final_stages = merge_and_order(merged_policy_stages, project_stages)

This simple reversal of merge order gives policy stages precedence while maintaining all dependency relationships through the DAG.

Use Cases

This enables organizations to:

• Run security scans (SonarQube, Semgrep, KICS) before project stages
• Enforce compliance checks at the beginning of pipelines
• Position policy jobs strategically without forcing them into .pipeline-policy-pre
• Reuse build artifacts in policy jobs without duplication

Changelog: added EE: true

References

Closes #526072 (closed)

Related:

• #479493 - Pipeline failure handling for policy jobs

How to set up and validate locally

1. Create a security policy management project with a Pipeline Execution Policy:

   ---
   pipeline_execution_policy:
   - name: Test Stage Injection
     description: 'Test new stage injection strategy'
     enabled: true
     pipeline_config_strategy: inject_policy
     content:
       include:
       - project: path/to/policy-project
         file: policy.yml
   experiments:
     pipeline_execution_policy_stages_higher_precedence:
       enabled: true

2. In the policy project, create policy.yml:

   stages:
   - .pipeline-policy-pre
   - .pre
   - policy-build
   - policy-test
   - policy-deploy
   - .post
   
   policy-build-job:
     stage: policy-build
     script: echo "Policy build"
   
   policy-test-job:
     stage: policy-test
     script: echo "Policy test"

3. In your target project, create .gitlab-ci.yml:

   stages:
   - build
   - test
   - deploy
   
   build-job:
     stage: build
     script: echo "Project build"
   
   test-job:
     stage: test
     script: echo "Project test"

4. Run a pipeline and verify stages are ordered as:

   [.pipeline-policy-pre, .pre, policy-build, policy-test, policy-deploy, build, test, deploy, .post]

5. Disable the experiment and verify stages revert to:

   [.pipeline-policy-pre, .pre, build, test, deploy, policy-build, policy-test, policy-deploy, .post]

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.