Backend: Multiple caches leading to a hash collision
Workaround
To avoid the possible "key" collisions provide a hardcoded key:prefix
as a solution.
test-job:
cache:
- key:
files:
- Gemfile.lock
prefix: '1'
paths:
- vendor/ruby
- key:
files:
- yarn.lock
prefix: '2'
paths:
- .yarn-cache/
Summary
Using multiple caches for a single project may lead to a cache collision due to how we are calculating the key.
Steps to reproduce
- Run the pipeline
- Create a
.gitlab-ci.yaml
that has multiple cache keys that depend on the lock file. - See cache key collision (I used
css
here as the code (```css) highlighter to outline the number of files and the cache key):
Creating cache 25b4b97e42e1c8264602627bd9591c7428c46135-1-non_protected...
node_modules/: found 844 matching files and directories
Creating cache 25b4b97e42e1c8264602627bd9591c7428c46135-1-non_protected...
folder2/node_modules/: found 2946 matching files and directories
Example Project
- Example Project: https://gitlab.com/gitlab-gold/faleksic-tests/public-group/ci-testing/-/tree/cache-collision
- Example Pipeline: https://gitlab.com/gitlab-gold/faleksic-tests/public-group/ci-testing/-/jobs/2461580650
What is the current bug behavior?
The cache uses the same key for different files.
Even though the package.json
is different in the packages it lists it still ends up as the same key.
What is the expected correct behavior?
Each key is unique.
Relevant logs and/or screenshots
Output of checks
This bug happens on GitLab.com
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
I believe the following line is calculating the key - https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/pipeline/seed/build/cache.rb#L55