## Status update (2022-10-19) - We have a [MR](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3613) ready to ship. When shipped (**target 15.7**), job variable expansion will be changed and handled as expected for variables of type = file. In summary, going forward, variables that reference/alias file variables are expanded to the file variable's file name/path *instead* of its value (i.e., the file's content). This is a **breaking change** but does resolve a potential security issue as secrets may be leaked with the current bug. - The plan is to simply ship the MR with the fix. There is no option as mentioned in some of the threads below, for shipping this fix behind a feature flag, providing a warning to the users. - I will keep the option open to pull the ship date into 15.6, as there are solid reasons, as outlined below, to do so. But I do want to give other users and customers that have not been following this issue the opportunity to be made aware of this change as there is a likelihood that the fix does, in fact impact a current workflow. Add comments to the new [comment thread](https://gitlab.com/gitlab-org/gitlab/-/issues/29407#note_1142232304) below. @DarrenEastman [Link](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3613#whats-the-best-way-to-test-this-mr) to before and after. ### Summary When using file variables in the "variables" section of a job in .gitlab-ci.yml the contents are being written to the new variable. ### Steps to reproduce The following gitlab-ci.yml ```yaml .deployment: image: alpine:latest script: - kubectl config set-cluster ${KUBE_CLUSTER} --server=${KUBE_SERVER} --certificate-authority="${KUBE_CA}" "Deploy to stage": extends: .deployment stage: deploy:stage when: manual variables: KUBE_CA: $KUBE_CA_STAGE "Deploy to prod": extends: .deployment stage: deploy:prod when: manual variables: KUBE_CA: $KUBE_CA_PROD ``` The variable KUBE_CA_PROD / KUBE_CA_STAGE are file variables in Gitlab. When using the variable KUBE_CA the contents are no longer a path to a file but the content of the file. ### What is the current *bug* behavior? Contents of file variables are written to variables ### What is the expected *correct* behavior? Path of file variable is used as variable #### Results of GitLab environment info <details> <summary>Expand for output related to GitLab environment info</summary> <pre> System information System: Ubuntu 18.04 Proxy: no Current User: git Using RVM: no Ruby Version: 2.5.3p105 Gem Version: 2.7.9 Bundler Version:1.17.3 Rake Version: 12.3.2 Redis Version: 3.2.12 Git Version: 2.21.0 Sidekiq Version:5.2.7 Go Version: unknown GitLab information Version: 11.11.2-ee Revision: cfdecb7c5de Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 9.6.11 URL: https://gitlab.XXXXX.com HTTP Clone URL: https://gitlab.XXXXX.com/some-group/some-project.git SSH Clone URL: git@gitlab.XXXXX.com:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: yes Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 9.1.0 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git </pre> </details> #### Results of GitLab application Check <details> <summary>Expand for output related to the GitLab application check</summary> <pre> not relevant </pre> </details> ### Proposal | Step | Status | | ------ | ------ | | 1. GitLab: Stop expanding file variables when sending to Runner | https://gitlab.com/gitlab-org/gitlab/-/issues/365859 | | 2. Runner: Refactor the creation of the job temporary file path | https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29128 | | **3. Fix file variables in Runner** | :point_left: You are here | #### Technical https://gitlab.com/gitlab-org/gitlab/-/issues/29407#note_935276800 ### Note TODO: Update the docs in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/75477 when this issue is closed. ## Solution Once https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3613 is merged, file variables and variables the reference file variables will work as follows: Having created a file variable via the GitLab UI named `A_FILE_VAR` with content `this is some super secret content`, the following job definition: ```yaml work: stage: test variables: REF_FILE_VAR: $A_FILE_VAR script: - echo $A_FILE_VAR - cat $A_FILE_VAR - echo $REF_FILE_VAR - cat $REF_FILE_VAR ``` will result in the following **before** and **after** outputs: ### Before ```sh $ echo $A_FILE_VAR /builds/avonbertoldi/test-project.tmp/A_FILE_VAR $ cat $A_FILE_VAR this is some super secret content $ echo $REF_FILE_VAR this is some super secret content $ cat $REF_FILE_VAR cat: can't open 'this': No such file or directory cat: can't open 'is': No such file or directory cat: can't open 'some': No such file or directory cat: can't open 'super': No such file or directory cat: can't open 'secret': No such file or directory cat: can't open 'content': No such file or directory ``` ### After ```sh $ echo $A_FILE_VAR /builds/avonbertoldi/test-project.tmp/A_FILE_VAR $ cat $A_FILE_VAR this is some super secret content $ echo $REF_FILE_VAR /builds/avonbertoldi/test-project.tmp/A_FILE_VAR $ cat $REF_FILE_VAR this is some super secret content ```