Non-standard DNS setups can cause failures in the test suite
Summary
Developers with non-standard DNS setups can trigger a number of errors in the spec suite. Known examples so far are myself and @nick.thomas. Both of us are running local DNS servers (I'm using a pi-hole, and Nick runs unbound), and both of us use the same ISP (not sure if that's related). I know that I, personally, also mess with any requests to Google's DNS servers (8.8.8.8, 8.8.4.4) and send them to the pi-hole, which then uses the ISP DNS servers as its upstream servers.
I believe this might actually be causing me an awful lot more spec failures across the whole suite, but the following example file is a particularly simple example of the issue.
Unsure of the correct labels so going with master:flaky for now, please adjust if needed.
Steps to reproduce
- Have a weird DNS setup (might need to extrapolate on this
😅 ) bundle exec rspec spec/lib/gitlab/url_blocker_spec.rb
What is the current bug behavior?
⋊> ~/D/g/g/gitlab on master ⨯ bundle exec rspec spec/lib/gitlab/url_blocker_spec.rb
Run options: include {:focus=>true}
All examples were filtered out; ignoring {:focus=>true}
==> Setting up Gitaly...
Checking gitaly-ruby Gemfile...
Checking gitaly-ruby bundle...
The Gemfile's dependencies are satisfied
Trying to connect to gitaly: ... OK
Gitaly set up in 0.56654986 seconds...
==> Setting up GitLab Elasticsearch Indexer...
GitLab Elasticsearch Indexer set up in 8.0851e-05 seconds...
...F..F........................FF..................................
Failures:
1) Gitlab::UrlBlocker#validate! when the URL hostname is a domain when domain cannot be resolved raises an error
Failure/Error: expect { subject }.to raise_error(described_class::BlockedUrlError)
expected Gitlab::UrlBlocker::BlockedUrlError but nothing was raised
# ./spec/lib/gitlab/url_blocker_spec.rb:62:in `block (5 levels) in <top (required)>'
2) Gitlab::UrlBlocker#validate! when the URL hostname is an IP address when the address is invalid raises an error
Failure/Error: expect { subject }.to raise_error(described_class::BlockedUrlError)
expected Gitlab::UrlBlocker::BlockedUrlError but nothing was raised
# ./spec/lib/gitlab/url_blocker_spec.rb:89:in `block (5 levels) in <top (required)>'
3) Gitlab::UrlBlocker#blocked_url? blocks urls with invalid ip address
Failure/Error: expect(described_class).to be_blocked_url('http://8.8.8.8.8')
expected `Gitlab::UrlBlocker.blocked_url?("http://8.8.8.8.8")` to return true, got false
# ./spec/lib/gitlab/url_blocker_spec.rb:622:in `block (3 levels) in <top (required)>'
4) Gitlab::UrlBlocker#blocked_url? blocks urls whose hostname cannot be resolved
Failure/Error: expect(described_class).to be_blocked_url('http://foobar.x')
expected `Gitlab::UrlBlocker.blocked_url?("http://foobar.x")` to return true, got false
# ./spec/lib/gitlab/url_blocker_spec.rb:628:in `block (3 levels) in <top (required)>'
Finished in 16.15 seconds (files took 17.23 seconds to load)
67 examples, 4 failures
Failed examples:
rspec ./spec/lib/gitlab/url_blocker_spec.rb:59 # Gitlab::UrlBlocker#validate! when the URL hostname is a domain when domain cannot be resolved raises an error
rspec ./spec/lib/gitlab/url_blocker_spec.rb:86 # Gitlab::UrlBlocker#validate! when the URL hostname is an IP address when the address is invalid raises an error
rspec ./spec/lib/gitlab/url_blocker_spec.rb:619 # Gitlab::UrlBlocker#blocked_url? blocks urls with invalid ip address
rspec ./spec/lib/gitlab/url_blocker_spec.rb:625 # Gitlab::UrlBlocker#blocked_url? blocks urls whose hostname cannot be resolved
According to @nick.thomas the following should be returning an empty array for the tests to run correctly:
[1] pry(main)> Addrinfo.getaddrinfo('foobar.x', 80, nil, :STREAM).to_a
=> [#<Addrinfo: [2001:8b0:1111:1111:0:ffff:51bb:ac50]:80 TCP (foobar.x)>, #<Addrinfo: 81.187.172.80:80 TCP (foobar.x)>]
What is the expected correct behavior?
Spec file should pass without errors
Results of GitLab environment info
Expand for output related to GitLab environment info
⋊> ~/D/g/g/gitlab on master ⨯ bundle exec rake gitlab:env:info 14:54:12System information System: Ubuntu 18.04 Proxy: no Current User: robotmay Using RVM: no Ruby Version: 2.6.3p62 Gem Version: 3.0.3 Bundler Version:1.17.3 Rake Version: 12.3.3 Redis Version: 4.0.9 Git Version: 2.24.1 Sidekiq Version:5.2.7 Go Version: go1.13.4 linux/amd64
GitLab information Version: 12.7.0-pre Revision: 719a29e873e Directory: /home/robotmay/Development/gitlab/gitlab-development-kit/gitlab DB Adapter: PostgreSQL DB Version: 10.10 URL: http://localhost:3000 HTTP Clone URL: http://localhost:3000/some-group/some-project.git SSH Clone URL: ssh://robotmay@localhost:2222/some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: no Using Omniauth: yes Omniauth Providers:
GitLab Shell Version: 10.3.0 Repository storage paths:
- default: / GitLab Shell path: /home/robotmay/Development/gitlab/gitlab-development-kit/gitlab-shell Git: /usr/bin/git
Possible fixes
Stub all DNS requests/run a mock DNS server with the test suite. This could also result in performance improvements by removing a big source of network requests.