GitLab Next

Summary

When using parent/child pipelines, variables are not passed to child according to documentation for priority of variables

Steps to reproduce

Gitlab-ce 13.5.3 (eaa194f15e6) ubuntu packages (Also in GitLab Enterprise Edition 13.4.3-ee as reported in #273420 (closed))

I conducted different tests by changing only the variable setup between child and parent pipeline.VAR is an example, can be anything. I tested various combinations between setting variable in schedule, parent pipeline and in bridge job calling the child pipeline. for each try I reported the values of the variable VAR, both in Parent and child Basically .gitlab-ci.yml has this structure:

variables:
  VAR: see below

stages:
  - job1
  - bridge

job1: 
 stage: job1
 script:
  - export

bridge:
  stage: bridge
  variables:
    VAR: see below
  trigger:
    include: child.yml

What is the current bug behavior?

With the setup above I performed the following tests:

1. No variable VAR defined in schedule, Parent: VAR: false, bridge: VAR not present -> job1 VAR=false, child VAR=false (as expected)
2. schedule: VAR=true, Parent: VAR: false, bridge: VAR: $VAR -> job1 VAR=true, child VAR=true (as expected)
3. schedule: VAR=true, Parent: VAR: false, bridge: VAR not present -> job1 VAR=true, child VAR=false (not really expected, I was expecting VAR in child to be true as schedule defined var should have priority)
4. schedule: VAR not present, Parent VAR: false, bridge: VAR: $VAR -> job1 VAR=false, child VAR="$VAR"

What is the expected correct behavior?

case 3: job1 VAR=true, child VAR=true
case 4: job1 VAR=false, child VAR=false (and not the weirdly escaped string)

Basically the combined effect of 3 and 4 prevents to have this behavior: a default value set globally in parent job, modifiable by schedule variable settings and picked up by child jobs. It seems that child job either gets wrong priority and/or fails to escape the variable.

Not sure about any workaround, maybe playing around with temporary variables or something like that. I can provide more information, but the gist is here.

Proposal

We have 2 different bugs here:

Bug 1.

3. schedule: VAR=true, Parent: VAR: false, bridge: VAR not present -> job1 VAR=true, child VAR=false (not really expected, I was expecting VAR in child to be true as schedule defined var should have priority)

from #281201 (comment 496235308):

This could be caused by the fact that the pipeline schedule object from the parent pipeline is not propagated to the child pipeline in Ci::Bridge#child_params, like we do for merge_request.

When we launched parent-child pipelines we only covered merge requests as MVC, the rest needed to be done.

I believe by passing schedule downstream in child_params, the child pipeline will have an associated schedule object and the variables will be automatically propagated. Same bug may exist with trigger_request since it's not propagated either. While we fix that we could also propagate external_pull_request for completeness.

Bug 2.

4. schedule: VAR not present, Parent VAR: false, bridge: VAR: $VAR -> job1 VAR=false, child VAR="$VAR"

from #281201 (comment 511203592):

This happens because the top-level variables: is used as default values when the job-level variables: doesn't specify it. We basically merge job-level variables over the top-level variables. Because the job/bridge-level variables: specifies VAR, then it takes precedence over the top-level VAR: foo. This makes the variable VAR: $VAR to be treated literally since $VAR is nowhere else defined.

variables:
  VAR: foo
  VAR2: bar

job:
  script: echo $VAR
  variables:
    VAR: $VAR

The above is equivalent to having:

job:
  script: echo $VAR
  variables:
    VAR: $VAR
    VAR2: bar

I think if we were expanding the job-level variables using the top-level variables prior to merging them, it should fix it. This way VAR: $VAR is expanded using {VAR: foo, VAR2: bar} as dictionary which becomes VAR: foo. Then when merged, it still takes precedence over the top-level variable.

Further details

Example project provided in duplicate issues

• #273420 (closed)
• #282475 (closed)