Artifact upload request timeout (HTTP 408) doesn't lead to failed CI build or dependency failures downstream

Summary

We had a CI pipeline push an artifact but fail silently. The downstream tests that relied on that artifact then failed in the script as the artifact files weren't there. Note that the job wasn't instantly failed with a dependency failure like when an artifact expires.

Steps to reproduce

Have a CI pipeline that uploads an artifact but have this time out somehow.

Example Project

Not sure how to reliably cause a timeout here. Maybe a stupidly large artifact and set the artifact limit to be really high?

What is the current bug behavior?

Gitlab doesn't consider an artifact upload timeout to be an error and so the job is marked as a success and downstream jobs just carry on working until they try to use the files from the artifact (as they aren't there) instead of getting a dependency failure on the job before it starts.

What is the expected correct behavior?

The job that failed to upload the artifact should have been marked as failed.

Relevant logs and/or screenshots

Build output:

...
Uploading artifacts...
cmake-build-debug/bin/*: found 234 matching files  
Job succeeded

Log line for the failed upload:

nginx/gitlab_access.log:10.7.245.0 - - [24/Oct/2019:13:46:58 +0000] "POST /api/v4/jobs/489720/artifacts?artifact_format=zip&artifact_type=archive&expire_in=2+days HTTP/1.1" 408 0 "" "gitlab-runner 12.4.0 (12-4-stable; go1.10.8; linux/amd64)"

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

Expand for output related to GitLab environment info

System information
System:		Ubuntu 18.04
Proxy:		no
Current User:	git
Using RVM:	no
Ruby Version:	2.6.3p62
Gem Version:	2.7.9
Bundler Version:1.17.3
Rake Version:	12.3.3
Redis Version:	3.2.12
Git Version:	2.22.0
Sidekiq Version:5.2.7
Go Version:	unknown
GitLab information
Version:	12.4.0-ee
Revision:	572e09f5e8f
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	9.6.14
URL:		https://gitlab.example.com
HTTP Clone URL:	https://gitlab.example.com/some-group/some-project.git
SSH Clone URL:	git@gitlab.example.com:some-group/some-project.git
Elasticsearch:	no
Geo:		no
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: google_oauth2
GitLab Shell
Version:	10.2.0
Repository storage paths:

• default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check

Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 10.2.0 ? ... OK (10.2.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Checking Reply by email ...
IMAP server credentials are correct? ... yes
Init.d configured correctly? ... skipped
MailRoom running? ... skipped
Checking Reply by email ... Finished
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
3/1 ... yes
3/7 ... yes
3/8 ... yes
30/9 ... yes
3/10 ... yes
2/11 ... yes
3/12 ... yes
3/13 ... yes
3/14 ... yes
3/15 ... yes
3/16 ... yes
3/17 ... yes
3/18 ... yes
3/19 ... yes
3/20 ... yes
3/21 ... yes
3/22 ... yes
3/23 ... yes
30/24 ... yes
3/25 ... yes
3/26 ... yes
3/27 ... yes
30/28 ... yes
3/30 ... yes
3/31 ... yes
3/32 ... yes
5/33 ... yes
10/34 ... yes
30/35 ... yes
3/36 ... yes
3/41 ... yes
30/42 ... yes
3/43 ... yes
10/44 ... yes
3/45 ... yes
3/46 ... yes
10/48 ... yes
10/50 ... yes
2/54 ... yes
10/55 ... yes
15/56 ... yes
15/58 ... yes
3/59 ... yes
3/60 ... yes
3/61 ... yes
3/64 ... yes
3/66 ... yes
30/68 ... yes
10/69 ... yes
11/70 ... yes
3/72 ... yes
18/74 ... yes
30/75 ... yes
10/76 ... yes
5/77 ... yes
16/78 ... yes
18/79 ... yes
3/80 ... yes
18/81 ... yes
19/82 ... yes
13/83 ... yes
13/84 ... yes
11/85 ... yes
15/86 ... yes
15/87 ... yes
15/88 ... yes
3/90 ... yes
16/91 ... yes
3/92 ... yes
3/93 ... yes
16/94 ... yes
30/95 ... yes
3/96 ... yes
16/97 ... yes
10/99 ... yes
3/101 ... yes
10/102 ... yes
18/103 ... yes
6/104 ... yes
3/105 ... yes
3/106 ... yes
3/107 ... yes
3/109 ... yes
3/110 ... yes
3/111 ... yes
10/112 ... yes
16/114 ... yes
10/115 ... yes
30/116 ... yes
2/117 ... yes
23/118 ... yes
3/119 ... yes
3/120 ... yes
3/121 ... yes
30/122 ... yes
16/123 ... yes
19/124 ... yes
13/125 ... yes
3/126 ... yes
3/127 ... yes
10/128 ... yes
10/129 ... yes
25/130 ... yes
25/131 ... yes
25/132 ... yes
16/133 ... yes
18/134 ... yes
11/135 ... yes
10/136 ... yes
2/138 ... yes
3/139 ... yes
30/141 ... yes
3/142 ... yes
13/147 ... yes
28/148 ... yes
28/149 ... yes
13/150 ... yes
19/151 ... yes
28/152 ... yes
28/153 ... yes
13/154 ... yes
13/155 ... yes
10/156 ... yes
11/157 ... yes
3/158 ... yes
13/159 ... yes
10/161 ... yes
13/162 ... yes
30/164 ... yes
13/165 ... yes
10/166 ... yes
3/167 ... yes
30/168 ... yes
30/169 ... yes
3/170 ... yes
30/172 ... yes
10/173 ... yes
10/174 ... yes
30/175 ... yes
3/176 ... yes
3/177 ... yes
28/179 ... yes
16/180 ... yes
10/181 ... yes
30/182 ... yes
30/183 ... yes
30/184 ... yes
31/185 ... yes
30/186 ... yes
38/187 ... yes
11/188 ... yes
14/190 ... yes
3/191 ... yes
13/192 ... yes
3/193 ... yes
13/194 ... yes
31/195 ... yes
31/196 ... yes
30/197 ... yes
30/198 ... yes
30/199 ... yes
3/200 ... yes
3/201 ... yes
26/202 ... yes
10/203 ... yes
28/204 ... yes
26/205 ... yes
26/206 ... yes
27/207 ... yes
26/208 ... yes
26/209 ... yes
10/210 ... yes
10/211 ... yes
10/212 ... yes
3/213 ... yes
3/214 ... yes
25/215 ... yes
31/217 ... yes
25/219 ... yes
13/220 ... yes
2/222 ... yes
3/223 ... yes
3/224 ... yes
10/225 ... yes
3/226 ... yes
19/227 ... yes
14/228 ... yes
18/229 ... yes
3/230 ... yes
3/231 ... yes
3/232 ... yes
26/233 ... yes
33/234 ... yes
26/235 ... yes
18/236 ... yes
36/238 ... yes
34/241 ... yes
3/242 ... yes
28/243 ... yes
32/244 ... yes
32/245 ... yes
38/246 ... yes
2/247 ... yes
26/248 ... yes
26/249 ... yes
26/250 ... yes
13/252 ... yes
5/253 ... yes
13/254 ... yes
34/255 ... yes
10/256 ... yes
14/257 ... yes
31/258 ... yes
16/259 ... yes
3/260 ... yes
2/261 ... yes
14/262 ... yes
13/263 ... yes
36/264 ... yes
10/265 ... yes
5/266 ... yes
3/267 ... yes
14/268 ... yes
14/270 ... yes
3/271 ... yes
32/272 ... yes
21/273 ... yes
36/274 ... yes
38/276 ... yes
44/277 ... yes
3/278 ... yes
3/279 ... yes
10/280 ... yes
10/281 ... yes
34/282 ... yes
3/283 ... yes
2/284 ... yes
34/285 ... yes
3/286 ... yes
30/287 ... yes
2/288 ... yes
49/289 ... yes
51/290 ... yes
13/292 ... yes
30/293 ... yes
10/294 ... yes
3/295 ... yes
3/296 ... yes
10/297 ... yes
36/298 ... yes
3/299 ... yes
3/300 ... yes
55/304 ... yes
55/305 ... yes
55/306 ... yes
55/307 ... yes
3/308 ... yes
10/309 ... yes
13/310 ... yes
3/311 ... yes
3/312 ... yes
14/313 ... yes
21/314 ... yes
13/315 ... yes
10/316 ... yes
3/317 ... yes
36/318 ... yes
14/319 ... yes
10/320 ... yes
21/321 ... yes
3/322 ... yes
14/323 ... yes
13/324 ... yes
23/325 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.3)
Git version >= 2.22.0 ? ... yes (2.22.0)
Git user has default SSH configuration? ... yes
Active users: ... 30
Is authorized keys file accessible? ... yes
Elasticsearch version 5.6 - 6.x? ... skipped (elasticsearch is disabled)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)