Passing-through a raw project/group/instance variable has unexpected result (#514239) · Issues · GitLab.org / GitLab

Passing-through a raw project/group/instance variable has unexpected result

<details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=514239) </details>  ### Summary When passing through a project/group/instance variable that is set to be `raw` as a job-level variable, you get the literal variable name as its value rather than the actual value. ### Steps to reproduce 1. Define a project level variable `MY_VARIABLE` * (It doesn't matter if it's project, group or instance level) * Value: `content_of_my_variable` * Masked: True (this is not functionally relevant, but can help telling things apart when debugging) * Expand variable reference: True 2. Add this `.gitlab-ci.yml`: ```yaml test: variables: MY_VARIABLE: value: $MY_VARIABLE script: - echo $MY_VARIABLE ``` 3. Run pipeline, observe `[MASKED]` in the output – meaning the project-level value is used as expected (and masked if we set up the variable as masked) 4. Change the project level variable to be **raw** (i.e. uncheck `Expand variable reference` in the UI) 5. Run pipeline again, observe `test` job now outputting the literal `$MY_VARIABLE` string This is wrong for **two** reasons: 1. The `raw` attribute should not make a difference here, because the project-level `MY_VARIABLE` does not contain a `$`. The job-level `MY_VARIABLE` is not defined as raw, so it should just be evaluated to the project-level value. 2. Bigger picture: The `script` should not even care about the variable being passed-through: Project level variables have precedence over YAML-defined variables. (Doing a variable pass-through doesn't really make sense in this scenario, you'd usually only do this on trigger jobs.) ### Example Project There are example pipelines showcasing this linked in [this internal investigation](https://gitlab.com/gitlab-com/request-for-help/-/issues/2124) ### What is the current *bug* behavior? When passing through a raw variable, the output for that variable becomes the literal variable name. ### What is the expected *correct* behavior? When passing through a raw variable, the output for that variable remains the variable value. ### Relevant logs and/or screenshots Looking at this debug output for relevant jobs leads is helpful to compare/understand: ```ruby job_id = 12345; build = ::Ci::Build.find(job_id); build.variables.to_hash ::Ci::BuildRunnerPresenter.new(build).runner_variables ``` Lots of debugging that me and Furkan already did is also available here internally: https://gitlab.com/gitlab-com/request-for-help/-/issues/2124#note_2280923616 As per https://gitlab.com/gitlab-org/gitlab/-/work_items/596367 the same behavior holds true when defining the variable via the top-level keyword instead of a job variable. ### Output of checks This bug happens on GitLab.com ### Possible fixes How we calculate [`scoped_variables`](https://gitlab.com/gitlab-org/gitlab/-/blob/075bc191faf785f0b1b24a547d26ce46744aee74/lib/gitlab/ci/variables/builder.rb#L28-29) might be relevant here. There's also a lengthy discussion about a similar issue with Runner expansion of empty variables here: https://gitlab.com/gitlab-org/gitlab/-/issues/350747#note_1265005773 – specifically how `JobVariables.Expand()` works be relevant here. It's not clear yet if a fix for this should happen on the Rails or the Runner side.

issue