Allow downstream pipelines to override predefined variables like GIT_SUBMODULE_STRATEGY

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem

When a project has GIT_SUBMODULE_STRATEGY configured as recursive, downstream pipelines cannot override this value to none in their own .gitlab-ci.yml configuration. This is due to how GitLab handles variable precedence and inheritance in downstream pipelines.

The issue affects ANY variable from upstream (predefined or user-defined) - downstream projects cannot override them.

How the Downstream Generator Works

Current Behavior

  1. Upstream bridge job triggers downstream pipeline: When a bridge job in the upstream project triggers a downstream pipeline, it calls Gitlab::Ci::Variables::Downstream::Generator.new(self).calculate

  2. Generator collects variables from bridge's scoped_variables: The generator includes the bridge's scoped_variables, which contain:

    • Predefined variables (like CI_PROJECT_ID, CI_COMMIT_SHA)
    • User-defined variables from .gitlab-ci.yml
    • Instance variables
    • Project variables
    • Group variables

  3. Variables passed as variables_attributes: These variables are passed to the downstream pipeline creation as variables_attributes in the target_revision parameters (see cross_project_params and child_params in the Bridge model)

  4. Highest precedence applied: Since these are "variables passed to downstream pipelines," they have the highest precedence in GitLab's variable hierarchy and cannot be overridden by the downstream project's own configuration

Why Downstream Project Variables Don't Work

The downstream project's .gitlab-ci.yml variables have lower precedence than the upstream-passed variables, so they cannot override them. This is by design to ensure upstream pipelines can reliably pass configuration to downstream pipelines, but it also prevents downstream projects from controlling their own behavior.

Root Cause

The Gitlab::Ci::Variables::Downstream::Generator includes the bridge's scoped_variables, which contain all variables from the upstream project:

  • Predefined variables (like CI_PROJECT_ID, CI_COMMIT_SHA)
  • User-defined variables from .gitlab-ci.yml
  • Project, group, and instance variables

These variables are passed to the downstream pipeline as variables_attributes with the highest precedence.

According to GitLab's variable precedence hierarchy:

  1. Variables passed to downstream pipelines (highest) ← Upstream variables end up here
  2. Trigger variables
  3. Scheduled pipeline variables
  4. Manual pipeline variables
  5. Variables added when creating a pipeline with the API
  6. Manual job variables
  7. Project variables (cannot override downstream-passed variables) ← Downstream variables end up here
  8. Group variables
  9. Instance variables
  10. Predefined variables (lowest)

This means that ANY variable from the upstream project automatically overrides the same variable in the downstream project, even though the downstream project should have control over its own behavior.

Expected Behavior

Downstream pipelines should be able to override variables in their own .gitlab-ci.yml configuration without requiring upstream configuration changes.

Current Workaround

The only current workaround is to explicitly override the variable in the upstream trigger job's variables section:

trigger_downstream:
  variables:
    GIT_SUBMODULE_STRATEGY: none
  trigger:
    project: downstream/project

This is not ideal because:

  1. It requires changes to the upstream project
  2. It couples the upstream and downstream projects tightly
  3. The downstream project cannot independently control its own behavior

Related Code

  • lib/gitlab/ci/variables/downstream/generator.rb - Calculates downstream variables
  • app/models/ci/bridge.rb - Bridge model that triggers downstream pipelines
  • lib/gitlab/ci/variables/builder.rb - Variables builder that determines precedence
Edited by 🤖 GitLab Bot 🤖