Enforce Custom Stages in Pipeline Execution Policies (#475152) · Issues · GitLab.org / GitLab

Enforce Custom Stages in Pipeline Execution Policies

# Release notes We're excited to introduce a new capability for Pipeline Execution Policies (PEP) that allows you to enforce **custom stages** into your CI/CD pipelines. This feature provides greater flexibility and control over your pipeline structure while maintaining security and compliance requirements. #### Benefits * **Enhanced pipeline customization**: Define and inject custom stages at specific points in your pipeline, allowing for more granular control over job execution order. * **Improved security and compliance**: Ensure that security scans and compliance checks run at the most appropriate times in your pipeline, such as after build but before deployment. * **Flexible policy management**: Maintain centralized policy control while allowing development teams to customize their pipelines within defined guardrails. * **Seamless integration**: Custom stages work alongside existing project stages and other policy types, providing a non-disruptive way to enhance your CI/CD workflows. #### How it works The new `inject_policy_ci` strategy for Pipeline Execution Policies allows you to define custom stages in your policy configuration. These stages are then intelligently merged with your project's existing stages using a Directed Acyclic Graph (DAG) algorithm, ensuring proper ordering and preventing conflicts. For example, you can now easily inject a custom security scanning stage between your build and deploy stages: ```yaml stages: [build, policy-test, deploy] policy-security-scan-job: stage: policy-test script: - run_security_script ``` This feature provides a powerful way to standardize and enforce security practices across your projects while maintaining the flexibility needed for diverse development workflows. The `inject_ci` mode will be deprecated, allowing you to opt into the `inject_policy_ci` mode. The `inject_policy_ci` mode will become the default when configuring policies with `Inject` in the policy editor. # Problem to solve  Currently, Pipeline Execution Policies only allow jobs to be placed in the following two stages: - `.pipeline-policy-pre` which always run at the start of the pipeline. - `.pipeline-policy-post`, which always runs at the end. In the experimental feature [Pipeline execution policy action](https://docs.gitlab.com/17.0/ee/user/application_security/policies/scan-execution-policies.html#pipeline-execution-policy-action) (no longer available), there as an additional stage (`.pipeline-policy-test`) which always ran after the `test` stage, but this behavior was not carried over to `Pipeline Execution Policies`. This limitation can affect the needs of some users (for example, those that practice trunk-based development and can't make use of [MRAP's](https://docs.gitlab.com/ee/user/application_security/policies/scan-result-policies.html)) who require more granular control over when policy-enforced jobs run, especially in relation to critical stages like testing and deployment. ## Example use case Consider a workflow in a trunk-based development project in which a `deploy` job must run _only_ if the Pipeline Execution Policy jobs are successful and find no vulnerabilities: ```mermaid graph LR A[build] --> B[test] --> C[analyze]--> D[deploy] ``` The `deploy` job should only run if the `test` jobs run properly and generate security artifacts which are consumed consumed by the `analyze` job in order to parse any detected vulnerabilities. If any vulnerabilities are found, the `analyze` job fails the pipeline which prevents the vulnerabilities from being deployed. Ideally, the `analyze` job would be enforced via a Pipeline Execution Policy; however, since neither of the two available stages run after the `test` job, they cannot fail the pipeline and prevent the changes to being deployed. ```mermaid graph LR A[.pipeline-policy-pre] --> B[Build] --> C[Test] --> D[Deploy] --> E[.pipeline-policy-post] ``` Notice that previously, the `.pipeline-policy-pre` stage helped achieving just that, but the feature does no longer implement this stage: ```mermaid graph LR A[.pipeline-policy-pre] --> B[Build] --> C[Test] --> D[.pipeline-policy-test] --> E[Deploy] --> F[.pipeline-policy-post] ``` # Proposal Introduce support for "custom stages" that can be placed in the middle of the pipeline by defining stages sequentially. For example, `policy-test` can be placed between `test` and `deploy` in the project pipeline: ``` stages: [test, policy-test, deploy] policy-test-job: stage: policy-test script: - sleep 1 ``` It is possible to specify any possible stages that projects might be using. Only stages with jobs will be present in the resulting pipeline. For example: ``` stages: [build, compile, test, check, policy-test, deploy, publish] policy-test-job: stage: policy-test script: - sleep 1 ``` In this case, `policy-test` will be placed after `build, compile, test, check` and before `deploy, publish`. If project was using `stages: [compile, check, publish]`, the resulting pipeline would contain stages: `[compile, check, policy-test, publish]`. If policy stages only specify the custom stage, this will be placed at the end of the pipeline, just before `.pipeline-policy-post` (if used): ``` stages: [policy-test] policy-test-job: stage: policy-test script: - sleep 1 ``` With default project stages, this results in `[build, test, deploy, policy-test]`. When used in a project without `.gitlab-ci.yml`, the behavior is the same as injecting into default stages. #### Behavior with other policies - It is additive to other policies and the project - When used together with `override_project_ci` policies, it merges into the override stages. #### New strategy Users can specify a new `pipeline_config_strategy` called `inject_policy_ci` which enables custom stages from the policy pipelines to be injected. It is made as a separate strategy so that users can opt-in and avoid breaking changes. `inject_ci` will be deprecated and customers will be encouraged to use the new strategy. #### Possible pipeline errors Depending on the policy stages configuration, the pipeline could fail with this strategy - in case the stages are defined as cyclic. Example: Policy stages: `[test, deploy]` Project stages: `[build, deploy, test]` In this example, `deploy` depends on `test` in the policy, but `test` depends on `deploy` in the project. This configuration results in a "cyclic dependencies" error: ![CleanShot_2025-01-20_at_16.55.20_2x](/uploads/34c4b09bb1e6e074177507e8ba511e60/CleanShot_2025-01-20_at_16.55.20_2x.png){width="585" height="122"}  ## Verification steps 1. Create a project 2. Create a policy with `pipeline_config_strategy: inject_policy` 3. Verify the various examples mentioned in the documentation: https://docs.gitlab.com/ee/user/application_security/policies/pipeline_execution_policies.html#stages-injection

issue