Add options to control variables override behavior to PEP (!186120) · Merge requests · GitLab.org / GitLab

What does this MR do and why?

This is a proposal to add variables control to pipeline execution policies.

Currently, the variable enforcement works by setting all YAML variables from the policies with the highest precedence. This ensures that the variables set in the policy CI YAML cannot be overridden.

The new, proposed approach replaces the "highest precedence" mechanism of enforcement with selective merge of user-defined variables:

Current:

flowchart LR
  policy[Policy YAML variables] --> user_vars[Merge user-defined variables] --> reapply_policy[Replace with policy YAML variables to enforce them] --> variables[Resulting variables]

New:

flowchart LR
  policy[Policy YAML variables] --> allowed{"Override allowed?"} -- true --> reject_user_vars[Reject specific user-defined variables] --> variables[Resulting variables]
  allowed -- false --> allow_user_vars[Allow specific user-defined variables] --> variables

Challenges with the current approach

It's all or nothing - there may be variables needed to customize policy-enforced jobs (e.g. container_scanning) that the project teams should be allowed to set, such as CS_IMAGE for the scanner to work properly
(#510867 (closed)) When using override_project_ci and including the project's .gitlab-ci.yml into the policy pipeline, all variables become policy variables and are set with the highest precedence. This includes any YAML variables found in .gitlab-ci.yml. This creates confusion with pre-filled variables specified in the project CI, because the values provided when the pipeline is run are ignored (&16430 (comment 2326124000)).
All user-defined variables are accepted by default. To lock them down for enhanced security, the policy has to list all the variables with their enforced values. There's no possibility to easily lock all by default.

New approach

Proposed schema in the policy.yml:

pipeline_execution_policy:
  - name: Variables
    description: ''
    enabled: true
    pipeline_config_strategy: inject_policy
    content:
      include:
        - project: gitlab-org/pep/variables
          file: variables.yml
    variables_override:
      allowed: false
      exceptions:
        - SAST_EXCLUDED_ANALYZERS
        - CS_IMAGE

A new variables_override option:

We decided to keep it simple and only provide global configuration, there's no per-job options.
Based on feedback from multiple customers who prefer to allow variables by default and then exclude/restrict only specific variables, I've tried the approach based on exceptions instead of purely focusing on allowlist.
- The difference in implementation is just a few lines of code, so having exceptions doesn't add too much overhead
For backwards compatibility, when variables_override is not provided, the current "highest precedence" for all policy YAML variables is applied
When the option is provided, the "highest precedence" is not applied
All user-defined variables (instance / group / project / run / schedule variables - all those that could change the behavior) are filtered using the defined exceptions.
- allowed: true -> every user-defined variable is permitted, but the variables listed in exceptions are ignored
- allowed: false -> every user-defined variable is ignored, except for the variables listed in exceptions. These are permitted.
This option provides solution for all the challenges with the current precedence mentioned above

Problems to solve from the epic vs. the new approach

I want to enforce a container security scan, where all of the rules/configuration are enforceable, but project teams are able to provide the path to a valid container.

The following configuration prevents all user-defined variables that could bypass the enforcement (such as SECRET_DETECTION_DISABLED, SAST_DISABLED, etc.) except for CS_IMAGE:

variables_override:
  allowed: false
  exceptions:
    - CS_IMAGE

I want to enforce security scan, but lock variables (and prevent them from being modified) for variables that may disable the scans or manipulate the results.

There are two options:

Lock down all variables using allowed: false, optionally allow exceptions (less flexible but more secure)
Keep variables open and lock the ones that could disable the scans (more flexible but less secure, newly added scanner configuration variables need to be added to the exceptions):

variables_override:
  allowed: true
  exceptions:
    - SECRET_DETECTION_DISABLED
    - SAST_DISABLED
    - DEPENDENCY_SCANNING_DISABLED
    - DAST_DISABLED
    - CONTAINER_SCANNING_DISABLED

I want to ensure policy job variables have the highest precedence, but that downstream project variables otherwise honor the CI precedence.

I don't think it's a goal to "ensure the highest precedence", but rather to ensure that policy job variables cannot get overridden by user-defined variables, while allowing the downstream project variables to be customized.

By changing approach to ignore user-defined variables instead of override the precedence, pipeline execution policies can be configured to have no impact on the downstream project variables:

with allowed: true (behavior as in compliance pipelines)
with allowed: false and listing all necessary project overrides in exceptions

I want to migrate from compliance pipelines with minimal/no impact to current variable precedence behaviors.

The migration would involve the following steps:

override_project_ci strategy (as suggested currently)
Include the project CI YAML in the policy (as suggested currently)
Use variables_override -> allowed: true to keep the same behavior with regards to the variable precedence and allow the project teams to customize all variables
(optionally) Lock certain variables using exceptions (for enhanced security)

References

Epic

Screenshots or screen recordings

`allowed: false` with `exceptions` (allowlist approach)

CleanShot_2025-04-01_at_09.33.51

`allowed: true` with `exceptions` (denylist approach)

CleanShot_2025-04-01_at_09.37.04

`override_project_ci` strategy allowing customization of `.gitlab-ci.yml` variables

CleanShot_2025-04-01_at_10.14.47

`override_project_ci` strategy allowing customization of `.gitlab-ci.yml` variables using group CI/CD variables

CleanShot_2025-04-01_at_10.46.40

How to set up and validate locally

Enable feature flag security_policies_optional_variables_control
Create a project

Create policy.yml:

variables:
  CS_IMAGE: 'to override by project'

policy-job:
  script:
    - echo "CS_IMAGE - $CS_IMAGE"

Create a new pipeline execution policy:

pipeline_execution_policy:
  - name: Variables
    description: ''
    enabled: true
    pipeline_config_strategy: inject_policy
    content:
      include:
        - project: <project-path>
          file: policy.yml
    variables_override:
      allowed: false
      exceptions:
        - CS_IMAGE

Set CS_IMAGE variable using the Settings -> CI/CD variables
Run a pipeline and verify the output of the variable

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Other concerns raised with regards to the compliance in general

CS_IMAGE and container_scanning assume only one container is deployed in the pipeline. We should have a more generic way of defining a policy, such as "any deployed containers should be scanned for vulnerabilities"
I can specify CS_IMAGE as an empty container and bypass the intended compliance checks

Summary and next steps

The proposed approach solves the challenges we have with the current mechanism for variables enforcement using highest precedence. It provides more predictable enforcement that is also customizable, allowing easier migration from the compliance pipelines. Having the new logic tied to a new policy configuration option allows us to bring this functionality without introducing breaking changes. The effort here doesn't prevent us from going for inputs as a more viable, long-term solution.

Feedback round with devopsverify
Add tests and merge this MR
Add documentation
Check implications for scheduled pipeline execution policies
Include the new option by default for new policies (behind feature flag)
Extend the policy editor to have UI for the variables_override option (behind feature flag)

Related to #520088 (closed)

Edited May 16, 2025 by Martin Cavoj

Add options to control variables override behavior to PEP