Concurrent runs not reproducible when using cached .venv
Summary
When doing concurrent pipeline runs and caching .venv folder from stage to stage, the first run is always green, but the subsequent runs, i.e. the ones that store the cache in the path builds/id/concurrent-id/.../.venv
where concurrent-id > 0
, will give an error that packages installed in venv cannot be found.
We are using poetry
to generate the .venv
folder but even in the subsequent stages when accessing packages directly in the path, i.e. builds/id/concurrent-id/.../.venv/bin/pip freeze
, the file not found error happens.
Is there some symlink to the original path somewhere when downloading the zipped cache?
Steps to reproduce
Example Project
image: python:3.7.0
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
POETRY_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pypoetry"
POETRY_VIRTUALENVS_IN_PROJECT: "true"
stages:
- build
- test
cache: &global-cache
key: "proj-$CI_COMMIT_REF_SLUG"
paths:
- .cache/pip
- .cache/pypoetry
- .venv
- poetry.lock
test:
stage: test
cache:
<<: *global-cache
policy: pull
script:
- poetry run pytest
build_package:
stage: build
cache:
<<: *global-cache
policy: pull-push
script:
- poetry install
What is the current bug behavior?
Concurrent pipeline runs are not reproducible, only the first run uses the cache folder as expected
What is the expected correct behavior?
Even in concurrent runs, the cache path for .venv folder is used correctly
Relevant logs and/or screenshots
$ pwd
/builds/7kEjm6Pz/1/proj
$ ls ./.venv/bin
...
pytest
...
$ make ci_test
poetry run pytest -m "not integration" tests --capture=no -rA -vv
FileNotFoundError
[Errno 2] No such file or directory: '/builds/7kEjm6Pz/1/proj/.venv/bin/pytest'
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)