Jobs with dependencies can be triggered while the artifacts needed are no longer available
Summary
Jobs with dependencies could be triggered while the artifacts generated by the dependency are no longer available because they are expired. This results in broken builds and confused developers. Because a deployment job which worked yesterday doesn't work anymore today, this could result in an broken deployment when rolling back to a previous release
Steps to reproduce
Simple gitlab-ci with 2 steps, 1 to build the artifact, 1 to deploy
build:
stage: build
script: composer install
artifacts:
paths:
- vendor
expire_in: 1 min
# yes this is an absurd low expire
test:
stage: test
script: test
dependencies:
- build
deploy:
stage: deploy
script: deploy
dependencies:
- build
What is the current bug behavior?
Retrying the deployment after a few minutes result in an deploy where foo.txt doesn't exists in case of php/composer this means that application is deployed without the vendor folder and could result in an broken production environment
What is the expected correct behavior?
i see 3 options, all valid
- build refused to be rebuild
- build fails on downloading artifacts, and the 'script' is not triggered, resulting
- build automatic starts the build which produces the artifacts
(What you should see instead)
Relevant logs and/or screenshots
Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
on runner03 (31dbfcf1)
Using Docker executor with image Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
on runner03 (31dbfcf1)
Using Docker executor with image dealerdirect/php:5.6 ...
Using docker image sha256:d8ec6ff2090a60f39603a9aff44f55f2d868fdad5be2c610f19f463cade35311 for predefined container...
Pulling docker image dealerdirect/php:5.6 ...
Using docker image dealerdirect/php:5.6 ID=sha256:f214a97ac4c1a228615768e6cb069dbe141709e1bc5ddc7c91c9f3cc55b1f063 for build container...
Running on runner-31dbfcf1-project-56-concurrent-0 via runner03
Fetching changes...
Removing vendor/
HEAD is now at XXXXXXXXXXXXXXXXXXXXXXXXXXX
From https://XXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXX..XXXXXX DEV-1298 -> origin/DEV-1298
XXXXXX..XXXXXX DEV-1607/xxxxxxxxxxxxxxxx -> origin/DEV-1607/xxxxxxxxxxxxxxxxxxxxxxx
XXXXXX..XXXXXX master -> origin/master
* [new tag] v0.0.37 -> v0.0.37
Checking out XXXXXX as DEV-1298...
Skipping Git submodules setup
$ eval $(ssh-agent -s)
Agent pid 11
<sanitize ssh setup>
$ dep deploy ${CI_ENVIRONMENT_NAME:-local}
➤ Executing task deploy:prepare
Warning: Permanently added the ECDSA host key for IP address 'XXX.XXX.XXX.XXX' to the list of known hosts.
✔ Ok
➤ Executing task deploy:lock
✔ Ok
➤ Executing task deploy:release
✔ Ok
➤ Executing task build:env
✔ Ok
➤ Executing task rsync:warmup
✔ Ok
➤ Executing task rsync
✔ Ok
➤ Executing task deploy:shared
✔ Ok
➤ Executing task deploy:writable
✔ Ok
➤ Executing task artisan:cache:clear
➤ Executing task deploy:failed
✔ Ok
[RuntimeException]
PHP Warning: require(/path/to/release/folder/releases/15/bootstrap/../ven
dor/autoload.php): failed to open stream: No such file or directory in /path/to/release/folder/releases/15/bootstrap/autoload.php on line 17
PHP Fatal error: require(): Failed opening required '/path/to/release/folder/releases/15/bootstrap/../vendor/autoload.php' (include_path='.:/usr/sha
re/pear:/usr/share/php') in /path/to/release/folder/releases/15/bootstrap/
autoload.php on line 17
deploy [-p|--parallel] [--no-hooks]
Output of checks
(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)
Results of GitLab environment info
System information
System: Ubuntu 16.04
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.3.3p222
Gem Version: 2.6.6
Bundler Version:1.13.7
Rake Version: 10.5.0
Redis Version: 3.2.5
Git Version: 2.11.1
Sidekiq Version:4.2.7
GitLab information
Version: 9.1.3-ee
Revision: e28218c
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: postgresql
DB Version: 9.6.1
URL: https://git.XXXXXXXXXX.XXX
HTTP Clone URL: https://git.XXXXXXXXXX.XXX/some-group/some-project.git
SSH Clone URL: git@git.XXXXXXXXXX.XXX:some-group/some-project.git
Elasticsearch: no
Geo: no
Using LDAP: no
Using Omniauth: yes
Omniauth Providers: github
GitLab Shell
Version: 5.0.2
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks
Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Checking GitLab Shell ...
GitLab Shell version >= 5.0.2 ? ... OK (5.0.2)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... no
User id for git: 998. Groupd id for git: 998
Try fixing it:
sudo chown -R git:git /var/opt/gitlab/git-data/repositories
For more information see:
doc/install/installation.md in section "GitLab Shell"
Please fix the error above and rerun the checks.
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
2/1 ... ok
10/2 ... ok
13/3 ... ok
2/4 ... ok
2/5 ... ok
6/6 ... ok
6/7 ... ok
16/8 ... ok
16/9 ... ok
16/10 ... ok
16/11 ... ok
16/12 ... ok
16/13 ... ok
16/14 ... ok
16/15 ... ok
10/16 ... ok
16/17 ... ok
16/18 ... ok
16/19 ... ok
16/20 ... ok
16/21 ... ok
16/22 ... ok
16/23 ... ok
16/24 ... ok
16/25 ... ok
16/26 ... ok
16/27 ... ok
16/28 ... ok
9/29 ... ok
9/30 ... ok
9/31 ... ok
9/32 ... ok
9/33 ... ok
9/34 ... ok
13/35 ... ok
6/36 ... ok
3/37 ... ok
8/38 ... ok
9/39 ... ok
9/40 ... ok
11/41 ... ok
8/42 ... ok
13/43 ... ok
9/44 ... ok
12/45 ... ok
2/46 ... ok
11/47 ... ok
11/48 ... ok
18/49 ... repository is empty
9/50 ... ok
9/51 ... ok
12/52 ... ok
10/53 ... ok
11/54 ... ok
12/56 ... ok
16/57 ... ok
15/58 ... ok
8/59 ... ok
9/60 ... ok
9/61 ... ok
9/62 ... ok
13/63 ... ok
6/65 ... ok
6/66 ... ok
6/67 ... ok
6/68 ... ok
12/69 ... ok
12/70 ... ok
11/72 ... ok
13/73 ... ok
12/74 ... ok
11/76 ... ok
13/77 ... ok
10/78 ... ok
9/79 ... ok
8/80 ... ok
2/82 ... ok
15/84 ... ok
10/85 ... ok
4/86 ... ok
28/87 ... ok
28/88 ... ok
8/89 ... ok
28/90 ... ok
10/91 ... ok
11/92 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
projects have namespace: ...
2/1 ... yes
10/2 ... yes
13/3 ... yes
2/4 ... yes
2/5 ... yes
6/6 ... yes
6/7 ... yes
16/8 ... yes
16/9 ... yes
16/10 ... yes
16/11 ... yes
16/12 ... yes
16/13 ... yes
16/14 ... yes
16/15 ... yes
10/16 ... yes
16/17 ... yes
16/18 ... yes
16/19 ... yes
16/20 ... yes
16/21 ... yes
16/22 ... yes
16/23 ... yes
16/24 ... yes
16/25 ... yes
16/26 ... yes
16/27 ... yes
16/28 ... yes
9/29 ... yes
9/30 ... yes
9/31 ... yes
9/32 ... yes
9/33 ... yes
9/34 ... yes
13/35 ... yes
6/36 ... yes
3/37 ... yes
8/38 ... yes
9/39 ... yes
9/40 ... yes
11/41 ... yes
8/42 ... yes
13/43 ... yes
9/44 ... yes
12/45 ... yes
2/46 ... yes
11/47 ... yes
11/48 ... yes
18/49 ... yes
9/50 ... yes
9/51 ... yes
12/52 ... yes
10/53 ... yes
11/54 ... yes
12/56 ... yes
16/57 ... yes
15/58 ... yes
8/59 ... yes
9/60 ... yes
9/61 ... yes
9/62 ... yes
13/63 ... yes
6/65 ... yes
6/66 ... yes
6/67 ... yes
6/68 ... yes
12/69 ... yes
12/70 ... yes
11/72 ... yes
13/73 ... yes
12/74 ... yes
11/76 ... yes
13/77 ... yes
10/78 ... yes
9/79 ... yes
8/80 ... yes
2/82 ... yes
15/84 ... yes
10/85 ... yes
4/86 ... yes
28/87 ... yes
28/88 ... yes
8/89 ... yes
28/90 ... yes
10/91 ... yes
11/92 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.3.3)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 2.7.3 ? ... yes (2.11.1)
Active users: 16
Checking GitLab ... Finished
Possible fixes
(If you can, link to the line of code that might be responsible for the problem)
label ~CI ~artifacts ~bug