Unexpected behavior for dotenv expansion in CI/CD pipelines
Summary
For certain jobs (see below), variables specified in artifacts:reports:dotenv
are not being expanded as expected. Using needs:job
to refer to the job stage where the dotenv
is being created fixes the issue, but the documentation doesn't mention this.
Is this working as expected?
- Yes! (update docs)
- No! (bug)
Originally reported by a GitLab Premium customer in this Zendesk ticket (internal only)
Steps to reproduce
Create or edit a .gitlab-ci.yml
file to add a job that writes variables to a dotenv
file, and then attempt to expand those variables in a subsequent job.
Example Project
https://gitlab.com/gitlab-gold/duncan/envdot (thanks @duncan_harris!)
What is the current bug behavior?
Variable expansion does not occur on all jobs
What is the expected correct behavior?
Variable expansion should occur on all jobs
Relevant logs and/or screenshots
Relevant .gitlab-ci.yml
file: https://gitlab.com/gitlab-gold/duncan/envdot/-/blob/552c4fc41993681365a3d7a6137d2d7dd369b04c/.gitlab-ci.yml
Variable did expand:
[0KRunning with gitlab-runner 15.6.0~beta.186.ga889181a (a889181a)[0;m
[0K on blue-2.shared.runners-manager.gitlab.com/default XxUrkriX[0;m
section_start:1675211149:resolve_secrets
[0K[0K[36;1mResolving secrets[0;m[0;m
section_end:1675211149:resolve_secrets
[0Ksection_start:1675211149:prepare_executor
[0K[0K[36;1mPreparing the "docker+machine" executor[0;m[0;m
[0KUsing Docker executor with image ruby:3.1 ...[0;m
[0KPulling docker image ruby:3.1 ...[0;m
[0KUsing docker image sha256:7c3335e5ff057dfa67eec96f86e8337f38913e0851a66d43ec0a08b8f2e0f98f for ruby:3.1 with digest ruby@sha256:925e65291232f485917a31ac422282cf2e1a39c02eda0e794135d1fb36e395c9 ...[0;m
section_end:1675211181:prepare_executor
[0Ksection_start:1675211181:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on runner-xxurkrix-project-43079140-concurrent-0 via runner-xxurkrix-shared-1675211112-d44d37da...
section_end:1675211184:prepare_script
[0Ksection_start:1675211184:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1m$ eval "$CI_PRE_CLONE_SCRIPT"[0;m
[32;1mFetching changes with git depth set to 20...[0;m
Initialized empty Git repository in /builds/gitlab-gold/duncan/envdot/.git/
[32;1mCreated fresh repository.[0;m
[32;1mChecking out 552c4fc4 as deeper_test...[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1675211185:get_sources
[0Ksection_start:1675211185:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0KUsing docker image sha256:7c3335e5ff057dfa67eec96f86e8337f38913e0851a66d43ec0a08b8f2e0f98f for ruby:3.1 with digest ruby@sha256:925e65291232f485917a31ac422282cf2e1a39c02eda0e794135d1fb36e395c9 ...[0;m
[32;1m$ echo "Test is $BUILD_VARIABLE"[0;m
Test is "value_from_setup_env"
section_end:1675211186:step_script
[0Ksection_start:1675211186:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1675211187:cleanup_file_variables
[0K[32;1mJob succeeded[0;m
Variable did not expand:
[0KRunning with gitlab-runner 15.6.0~beta.186.ga889181a (a889181a)[0;m
[0K on blue-1.shared.runners-manager.gitlab.com/default j1aLDqxS[0;m
section_start:1675211149:resolve_secrets
[0K[0K[36;1mResolving secrets[0;m[0;m
section_end:1675211149:resolve_secrets
[0Ksection_start:1675211149:prepare_executor
[0K[0K[36;1mPreparing the "docker+machine" executor[0;m[0;m
[0KUsing Docker executor with image ruby:3.1 ...[0;m
[0KPulling docker image ruby:3.1 ...[0;m
[0KUsing docker image sha256:7c3335e5ff057dfa67eec96f86e8337f38913e0851a66d43ec0a08b8f2e0f98f for ruby:3.1 with digest ruby@sha256:925e65291232f485917a31ac422282cf2e1a39c02eda0e794135d1fb36e395c9 ...[0;m
section_end:1675211178:prepare_executor
[0Ksection_start:1675211178:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on runner-j1aldqxs-project-43079140-concurrent-0 via runner-j1aldqxs-shared-1675211106-db217fdf...
section_end:1675211181:prepare_script
[0Ksection_start:1675211181:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1m$ eval "$CI_PRE_CLONE_SCRIPT"[0;m
[32;1mFetching changes with git depth set to 20...[0;m
Initialized empty Git repository in /builds/gitlab-gold/duncan/envdot/.git/
[32;1mCreated fresh repository.[0;m
[32;1mChecking out 552c4fc4 as deeper_test...[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1675211182:get_sources
[0Ksection_start:1675211182:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0KUsing docker image sha256:7c3335e5ff057dfa67eec96f86e8337f38913e0851a66d43ec0a08b8f2e0f98f for ruby:3.1 with digest ruby@sha256:925e65291232f485917a31ac422282cf2e1a39c02eda0e794135d1fb36e395c9 ...[0;m
[32;1m$ echo "Test is $BUILD_VARIABLE"[0;m
Test is
section_end:1675211183:step_script
[0Ksection_start:1675211183:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1675211184:cleanup_file_variables
[0K[32;1mJob succeeded[0;m
Example when the needs
keyword is used, showing the variable expansion now works as expected: https://gitlab.com/gitlab-gold/duncan/envdot/-/jobs/3693420098
Output of checks
This bug happens on GitLab.com