CI Lint returns HTTP500 when the CI file contains an include to an invalid address

Summary

CI Lint returns HTTP500 when the CI file contains an include to an invalid address

Steps to reproduce

We run our Gitlab instance in docker, and we have a reverse proxy in front of it. So the gitlab.company.tld resolves to a different IP address from the Gitlab's docker image than it resolves from anywhere else. And the reverse proxy terminates the HTTP connection.

Given we have an include in our .gitlab-ci.yaml file from this Gitlab instance:

include:
  remote: https://gitlab.company.tld/group/general-ci/-/templates/v1.yaml

When we lint this file the following error is in the logs:

Started POST "/<NAMESPACE>/<PROJECT_ID>/-/ci/lint" for 172.18.0.1 at 2020-05-15 08:32:11 +0000
Processing by Projects::Ci::LintsController#create as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "content"=>"[FILTERED]", "namespace_id"=>"<NAMESPACE>", "project_id"=>"<PROJECT_ID>"}
Completed 500 Internal Server Error in 21ms (ActiveRecord: 2.6ms | Elasticsearch: 0.0ms | Allocations: 12072)
  
Errno::ECONNREFUSED (Failed to open TCP connection to 172.18.0.3:443 (Connection refused - connect(2) for "172.18.0.3" port 443)):
  
lib/gitlab/http.rb:24:in `perform_request'
lib/gitlab/ci/config/external/file/remote.rb:33:in `fetch_remote_content'
lib/gitlab/ci/config/external/file/remote.rb:18:in `block in content'
lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize'
lib/gitlab/ci/config/external/file/remote.rb:18:in `content'
lib/gitlab/ci/config/external/file/base.rb:89:in `validate_content!'
lib/gitlab/ci/config/external/file/base.rb:72:in `validate!'
lib/gitlab/ci/config/external/file/base.rb:20:in `initialize'
lib/gitlab/ci/config/external/file/remote.rb:14:in `initialize'
lib/gitlab/ci/config/external/mapper.rb:82:in `new'
lib/gitlab/ci/config/external/mapper.rb:82:in `block in select_first_matching'
lib/gitlab/ci/config/external/mapper.rb:81:in `map'
lib/gitlab/ci/config/external/mapper.rb:81:in `select_first_matching'
lib/gitlab/ci/config/external/mapper.rb:37:in `map'
lib/gitlab/ci/config/external/mapper.rb:37:in `process'
lib/gitlab/ci/config/external/processor.rb:12:in `initialize'
lib/gitlab/ci/config.rb:77:in `new'
lib/gitlab/ci/config.rb:77:in `build_config'
lib/gitlab/ci/config.rb:64:in `expand_config'
lib/gitlab/ci/config.rb:25:in `initialize'
lib/gitlab/ci/yaml_processor.rb:34:in `new'
lib/gitlab/ci/yaml_processor.rb:34:in `new_with_validation_errors'
app/controllers/projects/ci/lints_controller.rb:11:in `create'
app/controllers/application_controller.rb:479:in `set_current_admin'
lib/gitlab/session.rb:11:in `with_session'
app/controllers/application_controller.rb:470:in `set_session_storage'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:464:in `set_locale'
lib/gitlab/error_tracking.rb:48:in `with_context'
app/controllers/application_controller.rb:559:in `sentry_context'
lib/gitlab/application_context.rb:52:in `block in use'
lib/gitlab/application_context.rb:52:in `use'
lib/gitlab/application_context.rb:20:in `with_context'
app/controllers/application_controller.rb:455:in `set_current_context'
lib/gitlab/request_profiler/middleware.rb:17:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/multipart.rb:124:in `call'
lib/gitlab/middleware/read_only/controller.rb:53:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:23:in `call'
config/initializers/fix_local_cache_middleware.rb:9:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:60:in `call'
lib/gitlab/middleware/release_env.rb:12:in `call'

And the Gitlab UI shows a generic 500 error page: image

What is the current bug behavior?

See in the previous section.

What is the expected correct behavior?

I'd rather see something like Can't fetch the included file from this address: gitlab.company.tld/...., because: Failed to open TCP connection to 172.18.0.3:443 (Connection refused - connect(2) for "172.18.0.3" port 443

Relevant logs and/or screenshots

See in the previous section

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

Expand for output related to GitLab environment info 
[root@gitlab ~]# docker exec -it docker_gitlab_1 gitlab-rake gitlab:env:info

System information
System:		
Current User:	git
Using RVM:	no
Ruby Version:	2.6.5p114
Gem Version:	2.7.10
Bundler Version:1.17.3
Rake Version:	12.3.3
Redis Version:	5.0.7
Git Version:	2.26.2
Sidekiq Version:5.2.7
Go Version:	unknown

GitLab information
Version:	12.10.3
Revision:	e01ceba1b49
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	11.7
URL:		https://gitlab.company.tld
HTTP Clone URL:	https://gitlab.company.tld/some-group/some-project.git
SSH Clone URL:	git@gitlab.company.tld:some-group/some-project.git
Using LDAP:	yes
Using Omniauth:	yes
Omniauth Providers: 

GitLab Shell
Version:	12.2.0
Repository storage paths:
- default: 	/var/opt/gitlab/git-data/repositories
GitLab Shell path:		/opt/gitlab/embedded/service/gitlab-shell
Git:		/opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab subtasks ...


Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 12.2.0 ? ... OK (12.2.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Checking Reply by email ...


IMAP server credentials are correct? ... yes
Init.d configured correctly? ... skipped
MailRoom running? ... skipped


Checking Reply by email ... Finished


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... Server: ldapmain
LDAP authentication... Anonymous. No bind_dn or password configured
LDAP users with access to your GitLab server (only showing the first 100 results)
User output sanitized. Found 100 users of 100 limit.


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... no
Try fixing it:
sudo chown -R git /var/opt/gitlab/gitlab-rails/uploads
sudo find /var/opt/gitlab/gitlab-rails/uploads -type f -exec chmod 0644 {} ;
sudo find /var/opt/gitlab/gitlab-rails/uploads -type d -not -path /var/opt/gitlab/gitlab-rails/uploads -exec chmod 0700 {} ;
For more information see:
doc/install/installation.md in section "GitLab"
Please fix the error above and rerun the checks.
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
5/1 ... yes
19/4 ... yes
19/5 ... yes
19/6 ... yes
19/7 ... yes
19/8 ... yes
21/9 ... yes
2/10 ... yes
6/11 ... yes
28/15 ... yes
19/16 ... yes
28/17 ... yes
28/19 ... yes
19/20 ... yes
19/22 ... yes
23/23 ... yes
2/24 ... yes
2/26 ... yes
19/30 ... yes
34/33 ... yes
34/34 ... yes
21/35 ... yes
28/36 ... yes
2/37 ... yes
19/38 ... yes
27/39 ... yes
19/40 ... yes
19/41 ... yes
2/42 ... yes
23/43 ... yes
2/44 ... yes
38/47 ... yes
70/48 ... yes
40/49 ... yes
28/50 ... yes
32/51 ... yes
21/52 ... yes
32/53 ... yes
31/55 ... yes
2/56 ... yes
23/57 ... yes
8/58 ... yes
4/59 ... yes
4/60 ... yes
44/62 ... yes
2/63 ... yes
8/64 ... yes
70/65 ... yes
8/66 ... yes
4/67 ... yes
46/68 ... yes
4/69 ... yes
6/74 ... yes
49/76 ... yes
31/77 ... yes
31/78 ... yes
8/79 ... yes
50/80 ... yes
50/81 ... yes
38/82 ... yes
19/85 ... yes
39/88 ... yes
54/91 ... yes
54/92 ... yes
18/93 ... yes
19/94 ... yes
19/95 ... yes
19/96 ... yes
19/97 ... yes
19/98 ... yes
19/99 ... yes
4/100 ... yes
40/101 ... yes
40/102 ... yes
19/103 ... yes
40/105 ... yes
39/106 ... yes
54/107 ... yes
22/108 ... yes
19/109 ... yes
19/110 ... yes
4/111 ... yes
4/113 ... yes
8/114 ... yes
4/115 ... yes
31/116 ... yes
4/117 ... yes
13/118 ... yes
4/120 ... yes
4/121 ... yes
13/122 ... yes
6/123 ... yes
4/124 ... yes
54/125 ... yes
43/126 ... yes
8/127 ... yes
40/128 ... yes
31/129 ... yes
13/130 ... yes
31/131 ... yes
4/132 ... yes
58/134 ... yes
4/135 ... yes
4/136 ... yes
23/137 ... yes
8/138 ... yes
8/140 ... yes
19/141 ... yes
31/144 ... yes
61/145 ... yes
28/146 ... yes
19/147 ... yes
61/150 ... yes
61/151 ... yes
19/152 ... yes
28/153 ... yes
19/154 ... yes
4/155 ... yes
62/157 ... yes
50/158 ... yes
50/159 ... yes
50/160 ... yes
19/161 ... yes
19/162 ... yes
31/163 ... yes
19/164 ... yes
23/165 ... yes
8/166 ... yes
19/168 ... yes
50/169 ... yes
19/170 ... yes
31/171 ... yes
2/172 ... yes
31/173 ... yes
64/174 ... yes
65/183 ... yes
31/184 ... yes
4/185 ... yes
68/186 ... yes
23/187 ... yes
31/188 ... yes
31/189 ... yes
21/191 ... yes
16/192 ... yes
18/193 ... yes
70/194 ... yes
19/195 ... yes
70/196 ... yes
70/197 ... yes
28/198 ... yes
13/200 ... yes
44/201 ... yes
18/202 ... yes
31/203 ... yes
70/204 ... yes
13/205 ... yes
19/206 ... yes
31/207 ... yes
19/208 ... yes
42/209 ... yes
70/210 ... yes
19/211 ... yes
70/212 ... yes
19/213 ... yes
19/214 ... yes
44/215 ... yes
19/216 ... yes
19/217 ... yes
2/218 ... yes
70/219 ... yes
19/220 ... yes
19/221 ... yes
70/223 ... yes
4/224 ... yes
44/225 ... yes
70/227 ... yes
19/228 ... yes
19/229 ... yes
19/230 ... yes
19/231 ... yes
70/232 ... yes
70/233 ... yes
75/234 ... yes
34/235 ... yes
34/236 ... yes
75/237 ... yes
75/238 ... yes
70/239 ... yes
70/242 ... yes
70/243 ... yes
70/245 ... yes
42/246 ... yes
50/247 ... yes
76/248 ... yes
75/249 ... yes
77/250 ... yes
78/251 ... yes
78/252 ... yes
78/253 ... yes
19/254 ... yes
78/255 ... yes
79/256 ... yes
34/257 ... yes
79/258 ... yes
70/259 ... yes
70/260 ... yes
19/261 ... yes
70/262 ... yes
82/264 ... yes
70/266 ... yes
83/267 ... yes
19/268 ... yes
85/269 ... yes
34/270 ... yes
70/271 ... yes
19/272 ... yes
70/273 ... yes
70/274 ... yes
4/275 ... yes
75/276 ... yes
83/278 ... yes
70/279 ... yes
19/280 ... yes
75/281 ... yes
70/282 ... yes
70/283 ... yes
94/284 ... yes
94/285 ... yes
94/286 ... yes
94/287 ... yes
94/288 ... yes
19/289 ... yes
70/290 ... yes
70/292 ... yes
70/293 ... yes
70/294 ... yes
70/295 ... yes
70/296 ... yes
19/297 ... yes
19/298 ... yes
94/300 ... yes
83/301 ... yes
83/302 ... yes
70/305 ... yes
21/309 ... yes
19/310 ... yes
70/311 ... yes
24/312 ... yes
58/313 ... yes
10/314 ... yes
70/315 ... yes
19/319 ... yes
19/321 ... yes
70/322 ... yes
19/323 ... yes
19/324 ... yes
5/325 ... yes
70/326 ... yes
101/327 ... yes
19/328 ... yes
94/329 ... yes
21/333 ... yes
70/334 ... yes
70/335 ... yes
70/336 ... yes
70/338 ... yes
70/339 ... yes
75/340 ... yes
4/341 ... yes
19/342 ... yes
70/343 ... yes
75/344 ... yes
43/345 ... yes
83/346 ... yes
105/347 ... yes
64/348 ... yes
70/349 ... yes
70/350 ... yes
19/351 ... yes
70/353 ... yes
70/354 ... yes
75/355 ... yes
83/356 ... yes
83/357 ... yes
94/358 ... yes
44/359 ... yes
106/360 ... yes
70/361 ... yes
21/362 ... yes
70/363 ... yes
19/364 ... yes
94/366 ... yes
19/367 ... yes
19/368 ... yes
70/369 ... yes
94/370 ... yes
70/371 ... yes
58/372 ... yes
70/373 ... yes
2/374 ... yes
64/375 ... yes
94/378 ... yes
75/379 ... yes
70/380 ... yes
113/381 ... yes
113/382 ... yes
19/383 ... yes
94/384 ... yes
19/385 ... yes
75/386 ... yes
19/387 ... yes
70/388 ... yes
19/389 ... yes
19/390 ... yes
19/391 ... yes
21/392 ... yes
101/393 ... yes
21/394 ... yes
19/395 ... yes
117/396 ... yes
50/397 ... yes
70/398 ... yes
16/399 ... yes
81/400 ... yes
58/401 ... yes
70/402 ... yes
47/403 ... yes
19/404 ... yes
19/405 ... yes
70/406 ... yes
19/407 ... yes
21/408 ... yes
81/409 ... yes
70/410 ... yes
94/411 ... yes
4/412 ... yes
70/413 ... yes
70/414 ... yes
4/415 ... yes
19/416 ... yes
103/417 ... yes
70/418 ... yes
4/419 ... yes
32/420 ... yes
70/421 ... yes
19/422 ... yes
8/423 ... yes
75/424 ... yes
122/425 ... yes
2/427 ... yes
124/428 ... yes
19/429 ... yes
118/430 ... yes
33/431 ... yes
19/432 ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.5)
Git version >= 2.22.0 ? ... yes (2.26.2)
Git user has default SSH configuration? ... yes
Active users: ... 87
Is authorized keys file accessible? ... yes


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

(If you can, link to the line of code that might be responsible for the problem)