Local pipeline configuration silently ignored when Byte Order Mark (BOM) \ufeff Unicode character included at start of .gitlab-ci.yml file
Summary
Some editors will include the \ufeff
(BOM) character at the beginning of files to indicate the file is encoded in big-endian UTF-16. When this is applied to a .gitlab-ci.yml
file the extra character is not visible in the pipeline editor.
The effect of having this character present in the local pipeline configuration file is that the call to the GraphQL getCiConfigData
endpoint to merge the local config with other included files does not actually merge the local content, and reports no errors.
The results in:
- the Pipeline Editor reports no errors when the current local settings are viewed (and not changes made)
- pipelines execute using included file content without applying local settings from
.gitlab-ci.yml
This leads to considerable confusion when local changes are made to the .gitlab-ci.yml
file, no errors are reported when viewing the file in the Pipeline Editor (until a change is made using the Editor, which appears to strip the BOM sequence) and pipelines continue to run without error using the contents of the included files without any of the custom changes applied.
(The presence of the BOM sequence does not appear to prevent the includes specified in the local config file from being processed correctly).
This issue occurs in gitlab.com
Similar effects of this bug have been reported in #353946 and #354026 but I created a new issue to include the details of the API request and to highlight that this led to considerable confusion and time spent debugging in a recent customer support ticket . If the development effort required to have the API request ignore a BOM sequence is determined to be low it would be great if this could be assigned a non-Backlog milestone.
Steps to reproduce
- Create a project an
include.yml
file as follows:
variables:
MY_VAR: "default_value"
stages:
- build
- test
build-job:
stage: build
script:
- echo "build-job"
- echo "MY_VAR=$MY_VAR"
unit-test-job:
stage: test
script:
- echo "unit-test-job"
- From a git client session create a
.gitlab-ci.yml
file with the following content and with a BOM prefix and push to the repo:
include:
local: include.yaml
variables:
MY_VAR: "custom_value"
The BOM prefix can be added in vim by typing :set bomb
before saving the file, and checked by running od -c .gitlab-ci.yml
:
$ od -c .gitlab.yml
0000000 357 273 277 i n c l u d e : \n l o
0000020 c a l : i n c l u d e . y a m
0000040 l \n \n v a r i a b l e s : \n
0000060 M Y _ V A R : " c u s t o m _
0000100 v a l u e " \n
0000107
- When the changes are pused to the repo, in GitLab, observe that a new pipeline runs that ignores the customised variable in the local config file. Also observe that the Pipeline Editor does not show any errors.
The underlying API request which behaves differently depending on whether the BOM sequence is present can be demonstrated as follows:
- Run the following GraphQL API request using
curl
which is based on the request details captured by the browser dev tools when the Pipeline Editor is accessed, and noting the\ufeff
BOM sequence sent in thecontent
variable:
curl 'https://gitlab.com/api/graphql' \
--request POST \
--compressed \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Content-Type: application/json' \
-H 'Accept: */*' \
-H 'PRIVATE-TOKEN: TOKEN' \
--data '{"operationName":"getCiConfigData","variables":{"projectPath":"jfarmiloe_ultimate_group/subgroup1/basic-cicd","sha":"5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050","content":"\ufeffinclude:\n local: include.yaml\n\nvariables:\n MY_VAR: \"custom_value\"\n"},"query":"query getCiConfigData($projectPath: ID!, $sha: String, $content: String!) {\n ciConfig(projectPath: $projectPath, sha: $sha, content: $content) {\n errors\n includes {\n location\n type\n blob\n raw\n __typename\n }\n mergedYaml\n status\n stages {\n ...PipelineStagesConnection\n __typename\n }\n __typename\n }\n}\n\nfragment PipelineStagesConnection on CiConfigStageConnection {\n nodes {\n name\n groups {\n nodes {\n name\n size\n jobs {\n nodes {\n name\n script\n beforeScript\n afterScript\n environment\n allowFailure\n tags\n when\n only {\n refs\n __typename\n }\n except {\n refs\n __typename\n }\n needs {\n nodes {\n name\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n}\n"}'
which returns a response indicating no errors and a mergedYaml
result containing just the included file content:
{"data":{"ciConfig":{"errors":[],"includes":[{"location":"include.yaml","type":"local","blob":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/blob/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","raw":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/raw/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","__typename":"CiConfigInclude"}],"mergedYaml":"---\nvariables:\n MY_VAR: default_value\nstages:\n- \".pre\"\n- build\n- test\n- deploy\n- \".post\"\nbuild-job:\n stage: build\n script:\n - echo \"build-job\"\n - echo \"MY_VAR=$MY_VAR\"\nunit-test-job:\n stage: test\n script:\n - echo \"unit-test-job\"\n","status":"VALID","stages":{"nodes":[{"name":"build","groups":{"nodes":[{"name":"build-job","size":1,"jobs":{"nodes":[{"name":"build-job","script":["echo \"build-job\"","echo \"MY_VAR=$MY_VAR\""],"beforeScript":[],"afterScript":[],"environment":null,"allowFailure":false,"tags":[],"when":"on_success","only":{"refs":["branches","tags"],"__typename":"CiConfigJobRestriction"},"except":null,"needs":{"nodes":[],"__typename":"CiConfigNeedConnection"},"__typename":"CiConfigJob"}],"__typename":"CiConfigJobConnection"},"__typename":"CiConfigGroup"}],"__typename":"CiConfigGroupConnection"},"__typename":"CiConfigStage"},{"name":"test","groups":{"nodes":[{"name":"unit-test-job","size":1,"jobs":{"nodes":[{"name":"unit-test-job","script":["echo \"unit-test-job\""],"beforeScript":[],"afterScript":[],"environment":null,"allowFailure":false,"tags":[],"when":"on_success","only":{"refs":["branches","tags"],"__typename":"CiConfigJobRestriction"},"except":null,"needs":{"nodes":[],"__typename":"CiConfigNeedConnection"},"__typename":"CiConfigJob"}],"__typename":"CiConfigJobConnection"},"__typename":"CiConfigGroup"}],"__typename":"CiConfigGroupConnection"},"__typename":"CiConfigStage"}],"__typename":"CiConfigStageConnection"},"__typename":"CiConfig"}}}
- Run the same
curl
command with the BOM sequence removed from.gitlab-ci.yml
:
curl 'https://gitlab.com/api/graphql' \
--request POST \
--compressed \
-H 'Accept-Encoding: gzip, deflate, br' \
-H 'Content-Type: application/json' \
-H 'Accept: */*' \
-H 'PRIVATE-TOKEN: TOKEN' \
--data '{"operationName":"getCiConfigData","variables":{"projectPath":"jfarmiloe_ultimate_group/subgroup1/basic-cicd","sha":"5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050","content":"include:\n local: include.yaml\n\nvariables:\n MY_VAR: \"custom_value\"\n"},"query":"query getCiConfigData($projectPath: ID!, $sha: String, $content: String!) {\n ciConfig(projectPath: $projectPath, sha: $sha, content: $content) {\n errors\n includes {\n location\n type\n blob\n raw\n __typename\n }\n mergedYaml\n status\n stages {\n ...PipelineStagesConnection\n __typename\n }\n __typename\n }\n}\n\nfragment PipelineStagesConnection on CiConfigStageConnection {\n nodes {\n name\n groups {\n nodes {\n name\n size\n jobs {\n nodes {\n name\n script\n beforeScript\n afterScript\n environment\n allowFailure\n tags\n when\n only {\n refs\n __typename\n }\n except {\n refs\n __typename\n }\n needs {\n nodes {\n name\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n }\n __typename\n}\n"}'
which now returns a response showing no errors and the mergedYaml
result containing the custom variable definition:
{"data":{"ciConfig":{"errors":["build-job job: chosen stage does not exist; available stages are .pre, test, .post"],"includes":[{"location":"include.yaml","type":"local","blob":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/blob/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","raw":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/raw/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","__typename":"CiConfigInclude"}],"mergedYaml":"---\nvariables:\n MY_VAR: custom_value\nstages:\n- \".pre\"\n- test\n- \".post\"\nbuild-job:\n stage: build\n script:\n - echo \"build-job\"\n - echo \"MY_VAR=$MY_VAR\"\nunit-test-job:\n stage: test\n script:\n - echo \"unit-test-job\"\n","status":"INVALID","stages":{"nodes":[],"__typename":"CiConfigStageConnection"},"__typename":"CiConfig"}}}
A similar test can be performed with a local content:
value containing an invalid stages:
setting with the required build stage missing, e.g.
"variables":{"projectPath":"jfarmiloe_ultimate_group/subgroup1/basic-cicd","sha":"5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050","content":"include:\n local: include.yaml\n\nvariables:\n MY_VAR: \"custom_value\"\n\nstages:\n - test\n\n"}
When the curl command is run with a BOM sequence included the result indicates no errors found and the mergedYaml
value does not contain the local configuration:
{"data":{"ciConfig":{"errors":[],"includes":[{"location":"include.yaml","type":"local","blob":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/blob/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","raw":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/raw/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","__typename":"CiConfigInclude"}],"mergedYaml":"---\nvariables:\n MY_VAR: default_value\nstages:\n- \".pre\"\n- build\n- test\n- deploy\n- \".post\"\nbuild-job:\n stage: build\n script:\n - echo \"build-job\"\n - echo \"MY_VAR=$MY_VAR\"\nunit-test-job:\n stage: test\n script:\n - echo \"unit-test-job\"\n","status":"VALID","stages":{"nodes":[{"name":"build","groups":{"nodes":[{"name":"build-job","size":1,"jobs":{"nodes":[{"name":"build-job","script":["echo \"build-job\"","echo \"MY_VAR=$MY_VAR\""],"beforeScript":[],"afterScript":[],"environment":null,"allowFailure":false,"tags":[],"when":"on_success","only":{"refs":["branches","tags"],"__typename":"CiConfigJobRestriction"},"except":null,"needs":{"nodes":[],"__typename":"CiConfigNeedConnection"},"__typename":"CiConfigJob"}],"__typename":"CiConfigJobConnection"},"__typename":"CiConfigGroup"}],"__typename":"CiConfigGroupConnection"},"__typename":"CiConfigStage"},{"name":"test","groups":{"nodes":[{"name":"unit-test-job","size":1,"jobs":{"nodes":[{"name":"unit-test-job","script":["echo \"unit-test-job\""],"beforeScript":[],"afterScript":[],"environment":null,"allowFailure":false,"tags":[],"when":"on_success","only":{"refs":["branches","tags"],"__typename":"CiConfigJobRestriction"},"except":null,"needs":{"nodes":[],"__typename":"CiConfigNeedConnection"},"__typename":"CiConfigJob"}],"__typename":"CiConfigJobConnection"},"__typename":"CiConfigGroup"}],"__typename":"CiConfigGroupConnection"},"__typename":"CiConfigStage"}],"__typename":"CiConfigStageConnection"},"__typename":"CiConfig"}}}
But when the curl command is run without the BOM sequence the expected error is returned:
{"data":{"ciConfig":{"errors":["build-job job: chosen stage does not exist; available stages are .pre, test, .post"],"includes":[{"location":"include.yaml","type":"local","blob":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/blob/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","raw":"https://gitlab.com/jfarmiloe_ultimate_group/subgroup1/basic-cicd/-/raw/5c6eb9a353a4d3941fa2d3c4c04dd3ab66ec2050/include.yaml","__typename":"CiConfigInclude"}],"mergedYaml":"---\nvariables:\n MY_VAR: custom_value\nstages:\n- \".pre\"\n- test\n- \".post\"\nbuild-job:\n stage: build\n script:\n - echo \"build-job\"\n - echo \"MY_VAR=$MY_VAR\"\nunit-test-job:\n stage: test\n script:\n - echo \"unit-test-job\"\n","status":"INVALID","stages":{"nodes":[],"__typename":"CiConfigStageConnection"},"__typename":"CiConfig"}}}
Example Project
What is the current bug behavior?
Inclusion of a BOM sequence in a .gitlab-ci.yml
file prevents local configuration items being merged into the included file content.
What is the expected correct behavior?
The presence of a BOM sequence should have no effect on how pipeline configuration is merged.
Relevant logs and/or screenshots
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\`sudo gitlab-rake gitlab:env:info\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\`) (For installations from source run and paste the output of: \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of: \\\\\\\\\\\\\\\`sudo gitlab-rake gitlab:check SANITIZE=true\\\\\\\\\\\\\\\`) (For installations from source run and paste the output of: \\\\\\\\\\\\\\\`sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true\\\\\\\\\\\\\\\`) (we will only investigate if the tests are passing)