GitLab Runner does not detect untracked files with Unicode characters in filenames
Summary
GitLab Runner is not always capable of detecting untracked file with Unicode characters in filenames.
It is caused by a way how GitLab Runner retrieves a list of tracked files, Runner uses following command line git ls-files -o
. An output of this command line depends on core.quotePath
option. By default Git escapes "unusual" characters, Runner takes escaped paths as it is, as result files with non-ASCII characters are not found.
Without explicit setting of core.quotePath
artifacts are not found: https://gitlab.com/kutelev/gost_forms/-/jobs/85540762
If core.quotePath
is explicitly set to false
all artifacts are found: https://gitlab.com/kutelev/gost_forms/-/jobs/85540626
On *NIX system such files are silently ignored and it is hard to understand why your files are not found. On Windows Runner raises complaints:
WARNING: "Merhaba d\303\274nya!": GetFileAttributesEx "Merhaba d/303/274nya!": The filename, directory name, or volume label syntax is incorrect.
WARNING: "\320\237\321\200\320\270\320\262\320\265\321\202, \320\274\320\270\321\200!": GetFileAttributesEx "/320/237/321/200/320/270/320/262/320/265/321/202, /320/274/320/270/321/200!": The filename, directory name, or volume label syntax is incorrect.
WARNING: "\343\203\217\343\203\255\343\203\274\343\203\273\343\203\257\343\203\274\343\203\253\343\203\211": GetFileAttributesEx "/343/203/217/343/203/255/343/203/274/343/203/273/343/203/257/343/203/274/343/203/253/343/203/211": The filename, directory name, or volume label syntax is incorrect.
Steps to reproduce
Just create any project which creates artifacts with Unicode characters in filenames and set untracked: true
in your .gitlab-ci.yml file.
Very likely such artifacts will be not found.
There is a simple project which creates 4 files:
- Hello, world!
- Привет, мир!
- Merhaba dünya!
- ハロー・ワールド'
With default GitLab Runner only first file is found (Hello, world!).
Actual behavior
GitLab Runner uses Git a way which can not guarantee detection of all untracked files.
Expected behavior
GitLab Runner must use Git a way which guarantees detection of all untracked files regardless core.quotePath
option value.
Relevant logs and/or screenshots
Environment description
The issue is reproducible with shared Runners on GitLab.com, the same problems happens on all desktop OSes including macOS and Windows.
Used GitLab Runner version
Running with gitlab-runner 11.1.0 (081978aa)
on docker-auto-scale 0277ea0f