GitLab says "Remote file: 'http://whatever.com/file.extension' is not valid" when it can't access the file for any reason

Summary

When GitLab can't access a remote file from an include directive for any reason, the following idiotic error message occurs:

From /<group>/<subgroup>/<project-name>/pipelines after clicking Create Pipeline:

image

Literal text:

The form contains the following error:
Remote file 'http://whatever.com/file.extension' is not valid.

(Remote file URL has been generalized to obscure the address.)

This code made a specific exception get generalized so that the real cause of the problem is completely hidden.

Steps to reproduce

Include a remote file that can't be downloaded for any reason:

  • Invalid URL
  • Unreachable website
  • etc...

Example Project

Don't need it.

What is the current bug behavior?

The Remote file is invalid error occurs when the remote file can't be downloaded for any reason.

What is the expected correct behavior?

The app should actually say what the problem is.

Relevant logs and/or screenshots

One of the configuration options that will make it possible to download remote files: image

But even with this, you can still get the same stupid error message for other reasons.

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

Expand for output related to GitLab environment info 
root@gitlab:/# gitlab-rake gitlab:env:info

System information
System:
Proxy:          LYRICAL_NGINX_SVC_SERVICE_PORT_PROXY: 444
Current User:   git
Using RVM:      no
Ruby Version:   2.4.4p296
Gem Version:    2.7.6
Bundler Version:1.16.2
Rake Version:   12.3.1
Redis Version:  3.2.11
Git Version:    2.18.0
Sidekiq Version:5.1.3
Go Version:     unknown


GitLab information
Version:        11.2.1-ee
Revision:       df8846c
Directory:      /opt/gitlab/embedded/service/gitlab-rails
DB Adapter:     postgresql
DB Version:     9.6.8
URL:            https://test-gitlab.bigt.corp
HTTP Clone URL: https://test-gitlab.bigt.corp/some-group/some-project.git
SSH Clone URL:  git@test-gitlab.bigt.corp:some-group/some-project.git
Elasticsearch:  no
Geo:            no
Using LDAP:     no
Using Omniauth: no


GitLab Shell
Version:        8.1.1
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
root@gitlab:/# gitlab-rake gitlab:check SANITIZE=true

Checking GitLab Shell ...


GitLab Shell version >= 8.1.1 ? ... OK (8.1.1)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
4/1 ... ok
4/2 ... ok
4/3 ... ok
8/5 ... ok
7/6 ... ok
4/7 ... ok
10/8 ... ok
4/9 ... ok
4/10 ... ok
4/11 ... ok
4/12 ... ok
4/13 ... ok
4/14 ... ok
4/15 ... ok
4/16 ... ok
17/17 ... ok
17/18 ... ok
4/19 ... ok
17/20 ... ok
10/21 ... ok
4/22 ... ok
4/23 ... ok
4/24 ... ok
4/25 ... ok
4/26 ... ok
4/27 ... ok
20/28 ... ok
10/29 ... ok
21/30 ... ok
17/31 ... ok
23/32 ... ok
10/33 ... ok
7/34 ... ok
21/35 ... ok
21/36 ... ok
10/37 ... ok
26/38 ... ok
27/39 ... repository is empty
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK


Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Sidekiq ...


Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Reply by email is disabled in config/gitlab.yml
Checking LDAP ...


LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
4/1 ... yes
4/2 ... yes
4/3 ... yes
8/5 ... yes
7/6 ... yes
4/7 ... yes
10/8 ... yes
4/9 ... yes
4/10 ... yes
4/11 ... yes
4/12 ... yes
4/13 ... yes
4/14 ... yes
4/15 ... yes
4/16 ... yes
17/17 ... yes
17/18 ... yes
4/19 ... yes
17/20 ... yes
10/21 ... yes
4/22 ... yes
4/23 ... yes
4/24 ... yes
4/25 ... yes
4/26 ... yes
4/27 ... yes
20/28 ... yes
10/29 ... yes
21/30 ... yes
17/31 ... yes
23/32 ... yes
10/33 ... yes
7/34 ... yes
21/35 ... yes
21/36 ... yes
10/37 ... yes
26/38 ... yes
27/39 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.5 ? ... yes (2.4.4)
Git version >= 2.9.5 ? ... yes (2.18.0)
Git user has default SSH configuration? ... yes
Active users: ... 11
Elasticsearch version 5.1 - 5.5? ... skipped (elasticsearch is disabled)


Checking GitLab ... Finished

Possible fixes

You need to write a reasonable error message in this code, so it actually shows the exception: https://gitlab.com/gitlab-org/gitlab-ee/commit/c73d0ff7bdcd#9ca378912ac316a047cd18073a593000ed651f82_20_20

Here's the line of code where the exceptions are being dropped on the floor without logging: https://gitlab.com/gitlab-org/gitlab-ee/commit/538e0cb1b0509e6804f1bdd13ccdbdbba4916d42#c39911f64c4e1aa2c87dad52dd31c825f3fd88f3_0_23

I fixed the code myself: nhooey/gitlab-ee@cc7dd0ca

Edited by 🤖 GitLab Bot 🤖