CI Lint returns HTTP500 when the CI file contains an include to an invalid address (#217926) · Issues · GitLab.org / GitLab

CI Lint returns HTTP500 when the CI file contains an include to an invalid address

### Summary CI Lint returns HTTP500 when the CI file contains an include to an invalid address ### Steps to reproduce We run our Gitlab instance in docker, and we have a reverse proxy in front of it. So the `gitlab.company.tld` resolves to a different IP address from the Gitlab's docker image than it resolves from anywhere else. And the reverse proxy terminates the HTTP connection. Given we have an `include` in our `.gitlab-ci.yaml` file from this Gitlab instance: ```ẏaml include: remote: https://gitlab.company.tld/group/general-ci/-/templates/v1.yaml ``` When we lint this file the following error is in the logs: ``` Started POST "/<NAMESPACE>/<PROJECT_ID>/-/ci/lint" for 172.18.0.1 at 2020-05-15 08:32:11 +0000 Processing by Projects::Ci::LintsController#create as HTML Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "content"=>"[FILTERED]", "namespace_id"=>"<NAMESPACE>", "project_id"=>"<PROJECT_ID>"} Completed 500 Internal Server Error in 21ms (ActiveRecord: 2.6ms | Elasticsearch: 0.0ms | Allocations: 12072) Errno::ECONNREFUSED (Failed to open TCP connection to 172.18.0.3:443 (Connection refused - connect(2) for "172.18.0.3" port 443)): lib/gitlab/http.rb:24:in `perform_request' lib/gitlab/ci/config/external/file/remote.rb:33:in `fetch_remote_content' lib/gitlab/ci/config/external/file/remote.rb:18:in `block in content' lib/gitlab/utils/strong_memoize.rb:30:in `strong_memoize' lib/gitlab/ci/config/external/file/remote.rb:18:in `content' lib/gitlab/ci/config/external/file/base.rb:89:in `validate_content!' lib/gitlab/ci/config/external/file/base.rb:72:in `validate!' lib/gitlab/ci/config/external/file/base.rb:20:in `initialize' lib/gitlab/ci/config/external/file/remote.rb:14:in `initialize' lib/gitlab/ci/config/external/mapper.rb:82:in `new' lib/gitlab/ci/config/external/mapper.rb:82:in `block in select_first_matching' lib/gitlab/ci/config/external/mapper.rb:81:in `map' lib/gitlab/ci/config/external/mapper.rb:81:in `select_first_matching' lib/gitlab/ci/config/external/mapper.rb:37:in `map' lib/gitlab/ci/config/external/mapper.rb:37:in `process' lib/gitlab/ci/config/external/processor.rb:12:in `initialize' lib/gitlab/ci/config.rb:77:in `new' lib/gitlab/ci/config.rb:77:in `build_config' lib/gitlab/ci/config.rb:64:in `expand_config' lib/gitlab/ci/config.rb:25:in `initialize' lib/gitlab/ci/yaml_processor.rb:34:in `new' lib/gitlab/ci/yaml_processor.rb:34:in `new_with_validation_errors' app/controllers/projects/ci/lints_controller.rb:11:in `create' app/controllers/application_controller.rb:479:in `set_current_admin' lib/gitlab/session.rb:11:in `with_session' app/controllers/application_controller.rb:470:in `set_session_storage' lib/gitlab/i18n.rb:55:in `with_locale' lib/gitlab/i18n.rb:61:in `with_user_locale' app/controllers/application_controller.rb:464:in `set_locale' lib/gitlab/error_tracking.rb:48:in `with_context' app/controllers/application_controller.rb:559:in `sentry_context' lib/gitlab/application_context.rb:52:in `block in use' lib/gitlab/application_context.rb:52:in `use' lib/gitlab/application_context.rb:20:in `with_context' app/controllers/application_controller.rb:455:in `set_current_context' lib/gitlab/request_profiler/middleware.rb:17:in `call' lib/gitlab/middleware/go.rb:20:in `call' lib/gitlab/etag_caching/middleware.rb:13:in `call' lib/gitlab/middleware/multipart.rb:124:in `call' lib/gitlab/middleware/read_only/controller.rb:53:in `call' lib/gitlab/middleware/read_only.rb:18:in `call' lib/gitlab/middleware/same_site_cookies.rb:27:in `call' lib/gitlab/middleware/basic_health_check.rb:25:in `call' lib/gitlab/middleware/request_context.rb:23:in `call' config/initializers/fix_local_cache_middleware.rb:9:in `call' lib/gitlab/metrics/requests_rack_middleware.rb:60:in `call' lib/gitlab/middleware/release_env.rb:12:in `call' ``` And the Gitlab UI shows a generic 500 error page: ![image](/uploads/a854df25a7efb0d18ece65ecd80ecc00/image.png) ### What is the current *bug* behavior? See in the previous section. ### What is the expected *correct* behavior? I'd rather see something like _Can't fetch the included file from this address: `gitlab.company.tld/....`, because: Failed to open TCP connection to 172.18.0.3:443 (Connection refused - connect(2) for "172.18.0.3" port 443_ ### Relevant logs and/or screenshots See in the previous section ### Output of checks (If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com) #### Results of GitLab environment info <details> <summary>Expand for output related to GitLab environment info</summary> <pre> [root@gitlab ~]# docker exec -it docker_gitlab_1 gitlab-rake gitlab:env:info System information System: Current User: git Using RVM: no Ruby Version: 2.6.5p114 Gem Version: 2.7.10 Bundler Version:1.17.3 Rake Version: 12.3.3 Redis Version: 5.0.7 Git Version: 2.26.2 Sidekiq Version:5.2.7 Go Version: unknown GitLab information Version: 12.10.3 Revision: e01ceba1b49 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 11.7 URL: https://gitlab.company.tld HTTP Clone URL: https://gitlab.company.tld/some-group/some-project.git SSH Clone URL: git@gitlab.company.tld:some-group/some-project.git Using LDAP: yes Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 12.2.0 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git </pre> </details> #### Results of GitLab application Check <details> <summary>Expand for output related to the GitLab application check</summary> <pre> Checking GitLab subtasks ... Checking GitLab Shell ... GitLab Shell: ... GitLab Shell version >= 12.2.0 ? ... OK (12.2.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Gitaly ... Gitaly: ... default ... OK Checking Gitaly ... Finished Checking Sidekiq ... Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking Incoming Email ... Incoming Email: ... Checking Reply by email ... IMAP server credentials are correct? ... yes Init.d configured correctly? ... skipped MailRoom running? ... skipped Checking Reply by email ... Finished Checking Incoming Email ... Finished Checking LDAP ... LDAP: ... Server: ldapmain LDAP authentication... Anonymous. No `bind_dn` or `password` configured LDAP users with access to your GitLab server (only showing the first 100 results) User output sanitized. Found 100 users of 100 limit. Checking LDAP ... Finished Checking GitLab App ... Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... no Try fixing it: sudo chown -R git /var/opt/gitlab/gitlab-rails/uploads sudo find /var/opt/gitlab/gitlab-rails/uploads -type f -exec chmod 0644 {} \; sudo find /var/opt/gitlab/gitlab-rails/uploads -type d -not -path /var/opt/gitlab/gitlab-rails/uploads -exec chmod 0700 {} \; For more information see: doc/install/installation.md in section "GitLab" Please fix the error above and rerun the checks. Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 5/1 ... yes 19/4 ... yes 19/5 ... yes 19/6 ... yes 19/7 ... yes 19/8 ... yes 21/9 ... yes 2/10 ... yes 6/11 ... yes 28/15 ... yes 19/16 ... yes 28/17 ... yes 28/19 ... yes 19/20 ... yes 19/22 ... yes 23/23 ... yes 2/24 ... yes 2/26 ... yes 19/30 ... yes 34/33 ... yes 34/34 ... yes 21/35 ... yes 28/36 ... yes 2/37 ... yes 19/38 ... yes 27/39 ... yes 19/40 ... yes 19/41 ... yes 2/42 ... yes 23/43 ... yes 2/44 ... yes 38/47 ... yes 70/48 ... yes 40/49 ... yes 28/50 ... yes 32/51 ... yes 21/52 ... yes 32/53 ... yes 31/55 ... yes 2/56 ... yes 23/57 ... yes 8/58 ... yes 4/59 ... yes 4/60 ... yes 44/62 ... yes 2/63 ... yes 8/64 ... yes 70/65 ... yes 8/66 ... yes 4/67 ... yes 46/68 ... yes 4/69 ... yes 6/74 ... yes 49/76 ... yes 31/77 ... yes 31/78 ... yes 8/79 ... yes 50/80 ... yes 50/81 ... yes 38/82 ... yes 19/85 ... yes 39/88 ... yes 54/91 ... yes 54/92 ... yes 18/93 ... yes 19/94 ... yes 19/95 ... yes 19/96 ... yes 19/97 ... yes 19/98 ... yes 19/99 ... yes 4/100 ... yes 40/101 ... yes 40/102 ... yes 19/103 ... yes 40/105 ... yes 39/106 ... yes 54/107 ... yes 22/108 ... yes 19/109 ... yes 19/110 ... yes 4/111 ... yes 4/113 ... yes 8/114 ... yes 4/115 ... yes 31/116 ... yes 4/117 ... yes 13/118 ... yes 4/120 ... yes 4/121 ... yes 13/122 ... yes 6/123 ... yes 4/124 ... yes 54/125 ... yes 43/126 ... yes 8/127 ... yes 40/128 ... yes 31/129 ... yes 13/130 ... yes 31/131 ... yes 4/132 ... yes 58/134 ... yes 4/135 ... yes 4/136 ... yes 23/137 ... yes 8/138 ... yes 8/140 ... yes 19/141 ... yes 31/144 ... yes 61/145 ... yes 28/146 ... yes 19/147 ... yes 61/150 ... yes 61/151 ... yes 19/152 ... yes 28/153 ... yes 19/154 ... yes 4/155 ... yes 62/157 ... yes 50/158 ... yes 50/159 ... yes 50/160 ... yes 19/161 ... yes 19/162 ... yes 31/163 ... yes 19/164 ... yes 23/165 ... yes 8/166 ... yes 19/168 ... yes 50/169 ... yes 19/170 ... yes 31/171 ... yes 2/172 ... yes 31/173 ... yes 64/174 ... yes 65/183 ... yes 31/184 ... yes 4/185 ... yes 68/186 ... yes 23/187 ... yes 31/188 ... yes 31/189 ... yes 21/191 ... yes 16/192 ... yes 18/193 ... yes 70/194 ... yes 19/195 ... yes 70/196 ... yes 70/197 ... yes 28/198 ... yes 13/200 ... yes 44/201 ... yes 18/202 ... yes 31/203 ... yes 70/204 ... yes 13/205 ... yes 19/206 ... yes 31/207 ... yes 19/208 ... yes 42/209 ... yes 70/210 ... yes 19/211 ... yes 70/212 ... yes 19/213 ... yes 19/214 ... yes 44/215 ... yes 19/216 ... yes 19/217 ... yes 2/218 ... yes 70/219 ... yes 19/220 ... yes 19/221 ... yes 70/223 ... yes 4/224 ... yes 44/225 ... yes 70/227 ... yes 19/228 ... yes 19/229 ... yes 19/230 ... yes 19/231 ... yes 70/232 ... yes 70/233 ... yes 75/234 ... yes 34/235 ... yes 34/236 ... yes 75/237 ... yes 75/238 ... yes 70/239 ... yes 70/242 ... yes 70/243 ... yes 70/245 ... yes 42/246 ... yes 50/247 ... yes 76/248 ... yes 75/249 ... yes 77/250 ... yes 78/251 ... yes 78/252 ... yes 78/253 ... yes 19/254 ... yes 78/255 ... yes 79/256 ... yes 34/257 ... yes 79/258 ... yes 70/259 ... yes 70/260 ... yes 19/261 ... yes 70/262 ... yes 82/264 ... yes 70/266 ... yes 83/267 ... yes 19/268 ... yes 85/269 ... yes 34/270 ... yes 70/271 ... yes 19/272 ... yes 70/273 ... yes 70/274 ... yes 4/275 ... yes 75/276 ... yes 83/278 ... yes 70/279 ... yes 19/280 ... yes 75/281 ... yes 70/282 ... yes 70/283 ... yes 94/284 ... yes 94/285 ... yes 94/286 ... yes 94/287 ... yes 94/288 ... yes 19/289 ... yes 70/290 ... yes 70/292 ... yes 70/293 ... yes 70/294 ... yes 70/295 ... yes 70/296 ... yes 19/297 ... yes 19/298 ... yes 94/300 ... yes 83/301 ... yes 83/302 ... yes 70/305 ... yes 21/309 ... yes 19/310 ... yes 70/311 ... yes 24/312 ... yes 58/313 ... yes 10/314 ... yes 70/315 ... yes 19/319 ... yes 19/321 ... yes 70/322 ... yes 19/323 ... yes 19/324 ... yes 5/325 ... yes 70/326 ... yes 101/327 ... yes 19/328 ... yes 94/329 ... yes 21/333 ... yes 70/334 ... yes 70/335 ... yes 70/336 ... yes 70/338 ... yes 70/339 ... yes 75/340 ... yes 4/341 ... yes 19/342 ... yes 70/343 ... yes 75/344 ... yes 43/345 ... yes 83/346 ... yes 105/347 ... yes 64/348 ... yes 70/349 ... yes 70/350 ... yes 19/351 ... yes 70/353 ... yes 70/354 ... yes 75/355 ... yes 83/356 ... yes 83/357 ... yes 94/358 ... yes 44/359 ... yes 106/360 ... yes 70/361 ... yes 21/362 ... yes 70/363 ... yes 19/364 ... yes 94/366 ... yes 19/367 ... yes 19/368 ... yes 70/369 ... yes 94/370 ... yes 70/371 ... yes 58/372 ... yes 70/373 ... yes 2/374 ... yes 64/375 ... yes 94/378 ... yes 75/379 ... yes 70/380 ... yes 113/381 ... yes 113/382 ... yes 19/383 ... yes 94/384 ... yes 19/385 ... yes 75/386 ... yes 19/387 ... yes 70/388 ... yes 19/389 ... yes 19/390 ... yes 19/391 ... yes 21/392 ... yes 101/393 ... yes 21/394 ... yes 19/395 ... yes 117/396 ... yes 50/397 ... yes 70/398 ... yes 16/399 ... yes 81/400 ... yes 58/401 ... yes 70/402 ... yes 47/403 ... yes 19/404 ... yes 19/405 ... yes 70/406 ... yes 19/407 ... yes 21/408 ... yes 81/409 ... yes 70/410 ... yes 94/411 ... yes 4/412 ... yes 70/413 ... yes 70/414 ... yes 4/415 ... yes 19/416 ... yes 103/417 ... yes 70/418 ... yes 4/419 ... yes 32/420 ... yes 70/421 ... yes 19/422 ... yes 8/423 ... yes 75/424 ... yes 122/425 ... yes 2/427 ... yes 124/428 ... yes 19/429 ... yes 118/430 ... yes 33/431 ... yes 19/432 ... yes Redis version >= 4.0.0? ... yes Ruby version >= 2.5.3 ? ... yes (2.6.5) Git version >= 2.22.0 ? ... yes (2.26.2) Git user has default SSH configuration? ... yes Active users: ... 87 Is authorized keys file accessible? ... yes Checking GitLab App ... Finished Checking GitLab subtasks ... Finished </pre> </details> ### Possible fixes (If you can, link to the line of code that might be responsible for the problem)

issue