Gitlab-ci cache directory not extracted
Problem:
I want to make pipeline quicker, since it takes ~10 minutes to execute it. In order to make it quicker, I want to cache project dependencies between pipelines (not only between jobs in the same pipeline)
What I tried:
I have tagged a runner to make sure, the pipeline is always executed on the same runner
I have configured gitlab-ci.yml to use this runner: runner-1
I have created a cache config (nested under job) as follows:
cache:
key: composer-cache-key
paths:
- composer-cache/
The job executes following script command COMPOSER_CACHE_DIR=composer-cache composer install --ignore-platform-reqs --no-progress. I can see, that composer-cache directory is successfully created (i can list the contents of the directory). However, subsequent pipeline execution still downloads dependencies from internet (instead of cache).
What I expect:
When the pipeline is executed for the first time, I expect a directory 'composer-cache' to be created (this works as expected). When the pipeline is executed subsequently, I expect this directory to be created after the cache is extracted. Although I can see in logs following message: Successfully extracted cache, I can not list the 'composer-cache' directory using the ls -l command prior composer install command is executed (which is wrong).
Config used:
image: docker:19
#services:
# - docker:19-dind
variables:
DOCKER_TLS_CERTDIR: ""
stages:
- build
- verify
- deploy
build:
image: awhalen/docker-php-composer-node:latest
stage: build
tags:
- runner-1
artifacts:
paths:
- vendor/
- composer.lock
expire_in: 1 day
# Select what we should cache
cache:
key: composer-cache-key
paths:
- composer-cache/
script:
- apt-get install screen
- COMPOSER_CACHE_DIR=composer-cache composer install --ignore-platform-reqs --no-progress
- pwd
- ls -l
Relevant logs:
Running with gitlab-runner 12.4.1 (05161b14)
on gitlab-runner-docker-1 _KvQRPgX
Using Docker executor with image awhalen/docker-php-composer-node:latest ...
00:02
WARNING: Container based cache volumes creation is disabled. Will not create volume for "/cache"
Using locally found image version due to if-not-present pull policy
Using docker image sha256:ff07a9a8a88d3241ded3cda3f6d8134c82265f9b3d1a2fbf7c4d1ef7a2d63f09 for awhalen/docker-php-composer-node:latest ...
...
Checking cache for composer-cache-key-2...
++ echo 'Checking cache for composer-cache-key-2...'
++ /usr/bin/gitlab-runner-helper cache-extractor --file ../../../cache/path/letmin/composer-cache-key-2/cache.zip --timeout 10
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
++ echo 'Successfully extracted cache'
+ exit 0
...
++ echo '$ COMPOSER_CACHE_DIR=composer-cache composer install --ignore-platform-reqs --no-progress'
++ COMPOSER_CACHE_DIR=composer-cache
++ composer install --ignore-platform-reqs --no-progress
$ COMPOSER_CACHE_DIR=composer-cache composer install --ignore-platform-reqs --no-progress
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Package operations: 147 installs, 0 updates, 0 removals
- Installing yiisoft/yii2-composer (2.0.9): Downloading (100%)
- Installing ezyang/htmlpurifier (v4.12.0): Downloading (100%)
- Installing cebe/markdown (1.2.1): Downloading (100%)
- Installing bower-asset/jquery (3.4.1): Downloading (100%)
...
$ ls -l
++ echo '$ pwd'
++ pwd
++ echo '$ ls -l'
++ ls -l
total 388
-rw-rw-rw- 1 root root 1622 Jun 12 12:39 LICENSE.md
-rw-rw-rw- 1 root root 3035 Jun 12 12:39 README.md
-rw-rw-rw- 1 root root 2624 Jun 12 12:39 Vagrantfile
drwxrwxrwx 12 root root 4096 Jun 12 12:39 backend
-rw-rw-rw- 1 root root 431 Jun 12 12:39 client_secret_379734329676-eg7lpgefcqipquk7le3e29tqb1rf6m2a.apps.googleusercontent.com.json
-rw-rw-rw- 1 root root 336 Jun 12 12:39 codeception.yml
drwxrwxrwx 11 root root 4096 Jun 12 12:39 common
drwxr-xr-x 5 root root 4096 Jun 12 12:40 composer-cache
-rw-rw-rw- 1 root root 3111 Jun 12 12:39 composer.json
-rw-rw-rw- 1 root root 284445 Jun 12 12:39 composer.lock
drwxrwxrwx 8 root root 4096 Jun 12 12:39 console
drwxrwxrwx 8 root root 4096 Jun 12 12:39 docker
-rw-rw-rw- 1 root root 1498 Jun 12 12:39 docker-compose-test.yml
-rw-rw-rw- 1 root root 1491 Jun 12 12:39 docker-compose.yml
drwxrwxrwx 5 root root 4096 Jun 12 12:39 environments
drwxrwxrwx 9 root root 4096 Jun 12 12:39 frontend
-rw-rw-rw- 1 root root 8660 Jun 12 12:39 init
-rw-rw-rw- 1 root root 319 Jun 12 12:39 init.bat
-rw-rw-rw- 1 root root 27 Jun 12 12:39 package-lock.json
-rw-rw-rw- 1 root root 5050 Jun 12 12:39 requirements.php
-rwxrwxrwx 1 root root 355 Jun 12 12:39 test.sh
drwxrwxrwx 7 root root 4096 Jun 12 12:39 tests
drwxrwxrwx 5 root root 4096 Jun 12 12:39 vagrant
drwxr-xr-x 55 root root 4096 Jun 12 12:41 vendor
-rw-rw-rw- 1 root root 323 Jun 12 12:39 yii.bat
...
Creating cache composer-cache-key-2...
++ echo 'Creating cache composer-cache-key-2...'
++ /usr/bin/gitlab-runner-helper cache-archiver --file ../../../cache/path/letmin/composer-cache-key-2/cache.zip --timeout 10 --path composer-cache/
composer-cache/: found 378 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
++ echo 'Created cache'
Created cache
...
++ /usr/bin/gitlab-runner-helper --version
++ echo 'Uploading artifacts...'
Uploading artifacts...--artifact-format zip --artifact-type archive
vendor/: found 26561 matching files
composer.lock: found 1 matching files
Uploading artifacts to coordinator... ok id=65485 responseStatus=201 Created token=mbMePKSV
+ exit 0
Job succeeded