Jobs with dependencies can be triggered while the artifacts needed are no longer available

Summary

Jobs with dependencies could be triggered while the artifacts generated by the dependency are no longer available because they are expired. This results in broken builds and confused developers. Because a deployment job which worked yesterday doesn't work anymore today, this could result in an broken deployment when rolling back to a previous release

Steps to reproduce

Simple gitlab-ci with 2 steps, 1 to build the artifact, 1 to deploy

build:
  stage: build
  script: composer install
  artifacts:
    paths:
      - vendor
    expire_in: 1 min 
# yes this is an absurd low expire

test:
  stage: test
  script: test
  dependencies:
  - build

deploy:
  stage: deploy
  script: deploy
  dependencies:
  - build

What is the current bug behavior?

Retrying the deployment after a few minutes result in an deploy where foo.txt doesn't exists in case of php/composer this means that application is deployed without the vendor folder and could result in an broken production environment

What is the expected correct behavior?

i see 3 options, all valid

build refused to be rebuild
build fails on downloading artifacts, and the 'script' is not triggered, resulting
build automatic starts the build which produces the artifacts

(What you should see instead)

Relevant logs and/or screenshots

Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
  on runner03 (31dbfcf1)
Using Docker executor with image Running with gitlab-ci-multi-runner 9.1.0 (0118d89)
  on runner03 (31dbfcf1)
Using Docker executor with image dealerdirect/php:5.6 ...
Using docker image sha256:d8ec6ff2090a60f39603a9aff44f55f2d868fdad5be2c610f19f463cade35311 for predefined container...
Pulling docker image dealerdirect/php:5.6 ...
Using docker image dealerdirect/php:5.6 ID=sha256:f214a97ac4c1a228615768e6cb069dbe141709e1bc5ddc7c91c9f3cc55b1f063 for build container...
Running on runner-31dbfcf1-project-56-concurrent-0 via runner03
Fetching changes...
Removing vendor/
HEAD is now at XXXXXXXXXXXXXXXXXXXXXXXXXXX
From https://XXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXX/XXXXXXXXXXXXXXXXXXXXXXXXXXX
   XXXXXX..XXXXXX  DEV-1298   -> origin/DEV-1298
   XXXXXX..XXXXXX  DEV-1607/xxxxxxxxxxxxxxxx -> origin/DEV-1607/xxxxxxxxxxxxxxxxxxxxxxx
   XXXXXX..XXXXXX  master     -> origin/master
 * [new tag]         v0.0.37    -> v0.0.37
Checking out XXXXXX as DEV-1298...
Skipping Git submodules setup
$ eval $(ssh-agent -s)
Agent pid 11
<sanitize ssh setup>
$ dep deploy ${CI_ENVIRONMENT_NAME:-local}
➤ Executing task deploy:prepare
Warning: Permanently added the ECDSA host key for IP address 'XXX.XXX.XXX.XXX' to the list of known hosts.
✔ Ok
➤ Executing task deploy:lock
✔ Ok
➤ Executing task deploy:release
✔ Ok
➤ Executing task build:env
✔ Ok
➤ Executing task rsync:warmup
✔ Ok
➤ Executing task rsync
✔ Ok
➤ Executing task deploy:shared
✔ Ok
➤ Executing task deploy:writable
✔ Ok
➤ Executing task artisan:cache:clear
➤ Executing task deploy:failed
✔ Ok

                                                                               
  [RuntimeException]                                                           
  PHP Warning:  require(/path/to/release/folder/releases/15/bootstrap/../ven  
  dor/autoload.php): failed to open stream: No such file or directory in /path/to/release/folder/releases/15/bootstrap/autoload.php on line 17           
  PHP Fatal error:  require(): Failed opening required '/path/to/release/folder/releases/15/bootstrap/../vendor/autoload.php' (include_path='.:/usr/sha  
  re/pear:/usr/share/php') in /path/to/release/folder/releases/15/bootstrap/  
  autoload.php on line 17                                                      
                                                                               

deploy [-p|--parallel] [--no-hooks]

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

System information
System:		Ubuntu 16.04
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.3.3p222
Gem Version:	2.6.6
Bundler Version:1.13.7
Rake Version:	10.5.0
Redis Version:	3.2.5
Git Version:	2.11.1
Sidekiq Version:4.2.7

GitLab information
Version:	9.1.3-ee
Revision:	e28218c
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
DB Version:	9.6.1
URL:		https://git.XXXXXXXXXX.XXX
HTTP Clone URL:	https://git.XXXXXXXXXX.XXX/some-group/some-project.git
SSH Clone URL:	git@git.XXXXXXXXXX.XXX:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: github

GitLab Shell
Version:	5.0.2
Repository storage paths:
- default: 	/var/opt/gitlab/git-data/repositories
Hooks:		/opt/gitlab/embedded/service/gitlab-shell/hooks
Git:		/opt/gitlab/embedded/bin/git

Results of GitLab application Check

Checking GitLab Shell ...

GitLab Shell version >= 5.0.2 ? ... OK (5.0.2)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... no
  User id for git: 998. Groupd id for git: 998
  Try fixing it:
  sudo chown -R git:git /var/opt/gitlab/git-data/repositories
  For more information see:
  doc/install/installation.md in section "GitLab Shell"
  Please fix the error above and rerun the checks.
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ... 
2/1 ... ok
10/2 ... ok
13/3 ... ok
2/4 ... ok
2/5 ... ok
6/6 ... ok
6/7 ... ok
16/8 ... ok
16/9 ... ok
16/10 ... ok
16/11 ... ok
16/12 ... ok
16/13 ... ok
16/14 ... ok
16/15 ... ok
10/16 ... ok
16/17 ... ok
16/18 ... ok
16/19 ... ok
16/20 ... ok
16/21 ... ok
16/22 ... ok
16/23 ... ok
16/24 ... ok
16/25 ... ok
16/26 ... ok
16/27 ... ok
16/28 ... ok
9/29 ... ok
9/30 ... ok
9/31 ... ok
9/32 ... ok
9/33 ... ok
9/34 ... ok
13/35 ... ok
6/36 ... ok
3/37 ... ok
8/38 ... ok
9/39 ... ok
9/40 ... ok
11/41 ... ok
8/42 ... ok
13/43 ... ok
9/44 ... ok
12/45 ... ok
2/46 ... ok
11/47 ... ok
11/48 ... ok
18/49 ... repository is empty
9/50 ... ok
9/51 ... ok
12/52 ... ok
10/53 ... ok
11/54 ... ok
12/56 ... ok
16/57 ... ok
15/58 ... ok
8/59 ... ok
9/60 ... ok
9/61 ... ok
9/62 ... ok
13/63 ... ok
6/65 ... ok
6/66 ... ok
6/67 ... ok
6/68 ... ok
12/69 ... ok
12/70 ... ok
11/72 ... ok
13/73 ... ok
12/74 ... ok
11/76 ... ok
13/77 ... ok
10/78 ... ok
9/79 ... ok
8/80 ... ok
2/82 ... ok
15/84 ... ok
10/85 ... ok
4/86 ... ok
28/87 ... ok
28/88 ... ok
8/89 ... ok
28/90 ... ok
10/91 ... ok
11/92 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
Send ping to redis server: OK
gitlab-shell self-check successful

Checking GitLab Shell ... Finished

Checking Sidekiq ...

Running? ... yes
Number of Sidekiq processes ... 1

Checking Sidekiq ... Finished

Checking Reply by email ...

Reply by email is disabled in config/gitlab.yml

Checking Reply by email ... Finished

Checking LDAP ...

LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab ...

Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config outdated? ... no
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory setup correctly? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
projects have namespace: ... 
2/1 ... yes
10/2 ... yes
13/3 ... yes
2/4 ... yes
2/5 ... yes
6/6 ... yes
6/7 ... yes
16/8 ... yes
16/9 ... yes
16/10 ... yes
16/11 ... yes
16/12 ... yes
16/13 ... yes
16/14 ... yes
16/15 ... yes
10/16 ... yes
16/17 ... yes
16/18 ... yes
16/19 ... yes
16/20 ... yes
16/21 ... yes
16/22 ... yes
16/23 ... yes
16/24 ... yes
16/25 ... yes
16/26 ... yes
16/27 ... yes
16/28 ... yes
9/29 ... yes
9/30 ... yes
9/31 ... yes
9/32 ... yes
9/33 ... yes
9/34 ... yes
13/35 ... yes
6/36 ... yes
3/37 ... yes
8/38 ... yes
9/39 ... yes
9/40 ... yes
11/41 ... yes
8/42 ... yes
13/43 ... yes
9/44 ... yes
12/45 ... yes
2/46 ... yes
11/47 ... yes
11/48 ... yes
18/49 ... yes
9/50 ... yes
9/51 ... yes
12/52 ... yes
10/53 ... yes
11/54 ... yes
12/56 ... yes
16/57 ... yes
15/58 ... yes
8/59 ... yes
9/60 ... yes
9/61 ... yes
9/62 ... yes
13/63 ... yes
6/65 ... yes
6/66 ... yes
6/67 ... yes
6/68 ... yes
12/69 ... yes
12/70 ... yes
11/72 ... yes
13/73 ... yes
12/74 ... yes
11/76 ... yes
13/77 ... yes
10/78 ... yes
9/79 ... yes
8/80 ... yes
2/82 ... yes
15/84 ... yes
10/85 ... yes
4/86 ... yes
28/87 ... yes
28/88 ... yes
8/89 ... yes
28/90 ... yes
10/91 ... yes
11/92 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.1.0 ? ... yes (2.3.3)
Your git bin path is "/opt/gitlab/embedded/bin/git"
Git version >= 2.7.3 ? ... yes (2.11.1)
Active users: 16

Checking GitLab ... Finished

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)

label ~~CI gitlab-ce~~2677571 gitlab-ce~2278648