Investigation Backend: Global variables must be explicitly forwarded to nested child pipeline, which overrides job variables

Summary

Documentation states that global variables are passed to child pipelines implicitly:

• https://docs.gitlab.com/ee/ci/pipelines/downstream_pipelines.html#pass-yaml-defined-cicd-variables states that by defining global variables in the parent CI config, they are automatically passed to a child pipeline, where they become 'trigger variables': "all jobs in a pipeline, including trigger jobs, inherit global variables"
• https://docs.gitlab.com/ee/ci/variables/index.html#cicd-variable-precedence lists 'trigger variables' at the very top of the variable precedence chain - above even global variables.
• https://docs.gitlab.com/ee/ci/triggers/index.html#pass-cicd-variables-in-the-api-call implies that 'trigger variables' have the highest priority of all variables and will always be passed to everything: "These variables have the highest precedence, and override all variables with the same name."

Therefore, when passing variables from a parent to a child pipeline, I expect those variables to also be passed to children of that child pipeline.

However, when starting a nested child pipeline, global variables defined in the parent are not passed to it.

There is a workaround: unless the nested child pipeline is triggered with forward: pipeline_variables: true - which documentation states is reserved for "manual pipeline variables and scheduled pipeline variables" - not global or trigger variables. (https://docs.gitlab.com/ee/ci/yaml/index.html#triggerforward)

However, there is an issue with that workaround. Job-level variables are expected to take precedence over global variables. When the intermediate child defines job-level variables but also explicitly passes global variables with forward: pipeline_variables, the forwarded global/trigger variables erroneously take precedence over the job-level variables. There is no workaround for this - as far as I know there is no way to override the global variable once forwarded.

Steps to reproduce

Create 5 jobs in 3 pipelines.

• job1 defines GLOBAL_VAR and creates a pipeline containing job2, job2-forward and job2-forward-new-value, which both create pipelines containing job3.
  • job2 calls job3 without explicitly forwarding any variables.
  • job2-forward calls job3 and explicitly forwards pipeline_variables.
  • job2-forward-new-value calls job3, reassigns GLOBAL_VAR with a job variable intended to override it, and explicitly forwards pipeline_variables.
• job3 reports the value of GLOBAL_VAR.
  • In job2 -> job3, GLOBAL_VAR is not set. ❌ Unexpected result.
  • In job2-forward -> job3, GLOBAL_VAR is set. ✅ Workaround.
  • In job2-forward-new-value -> job3, GLOBAL_VAR is set to the original value, not to the override. ❌ Unexpected result with no workaround.

# .gitlab-ci.yml
variables:
  GLOBAL_VAR: 'true'

job1:
  trigger:
    include: child.gitlab-ci.yml

# child.gitlab-ci.yml

# This triggered pipeline will fail
job2:
  trigger:
    include: nested-child.gitlab-ci.yml

# This triggered pipeline will pass
job2-forward:
  trigger:
    include: nested-child.gitlab-ci.yml
    forward:
      pipeline_variables: true

# This triggered pipeline will pass, but is expected to fail
job2-forward-new-value:
  variables:
    GLOBAL_VAR: 'false'
  trigger:
    include: nested-child.gitlab-ci.yml
    forward:
      pipeline_variables: true

# nested-child.gitlab-ci.yml
job3:
  script:
    - echo "The value of GLOBAL_VAR is $GLOBAL_VAR"
    # Fail the job if the global var is not set
    - test "$GLOBAL_VAR" = "true"

Example Project

https://gitlab.com/rossjrw/nested-child-pipeline-test

See pipeline run: https://gitlab.com/rossjrw/nested-child-pipeline-test/-/pipelines/829069022

Note that job2 -> job3 failed, job2-forward -> job3 passed, and job2-forward-new-value -> job3 passed (but should have failed).

What is the current bug behavior?

1. Global variables are not recursively passed to child pipelines. Explicitly forwarding pipeline_variables is required.
2. Job variables do not override global variables when pipeline_variables are forwarded.

What is the expected correct behavior?

For bug behaviour 1:

Global variables should be recursively passed to child pipelines.

OR

Documentation should be clear in that a) a parent's global variables become a child's trigger variables, b) a parent's trigger variables are considered manual variables for the purposes of variable forwarding.

For bug behaviour 2: Job-level variables should be passed to downstream pipelines with a higher priority than trigger variables passed via pipeline_variables forwarding.

Output of checks

This bug happens on GitLab.com