GitLab won't start if InfluxDB hostname cannot be resolved
Summary
I make last upgrade to 8.13.5, in my previous version i enable InfluxDB metrics, and configure. I running gitlab in docker and influxDB is a docker, and two container are linked.
Steps to reproduce
Enable and configure InfluxDB, stop gitlab, remove container and restart but remove --link to influxDB container.
Expected behavior
Make a warring in logs, and continue starting process.
Actual behavior
Gitlab not start get an 500 error.
Relevant logs and/or screenshots
==> /var/log/gitlab/sidekiq/current <==
2016-11-10_17:16:11.01087 getaddrinfo: Name or service not known
2016-11-10_17:16:11.01089 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/writer/udp.rb:13:in `connect'
2016-11-10_17:16:11.01090 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/writer/udp.rb:13:in `initialize'
2016-11-10_17:16:11.01090 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/client.rb:64:in `new'
2016-11-10_17:16:11.01090 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/client.rb:64:in `initialize'
2016-11-10_17:16:11.01090 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics.rb:149:in `new'
2016-11-10_17:16:11.01091 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics.rb:149:in `block in <module:Metrics>'
2016-11-10_17:16:11.01092 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:169:in `try_create'
2016-11-10_17:16:11.01092 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:81:in `block (2 levels) in pop'
2016-11-10_17:16:11.01092 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:77:in `loop'
2016-11-10_17:16:11.01093 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:77:in `block in pop'
2016-11-10_17:16:11.01093 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:76:in `synchronize'
2016-11-10_17:16:11.01093 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:76:in `pop'
2016-11-10_17:16:11.01094 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:89:in `checkout'
2016-11-10_17:16:11.01094 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:61:in `block in with'
2016-11-10_17:16:11.01094 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `handle_interrupt'
2016-11-10_17:16:11.01094 /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `with'
2016-11-10_17:16:11.01094 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics.rb:44:in `submit_metrics'
2016-11-10_17:16:11.01095 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:54:in `flush'
2016-11-10_17:16:11.01095 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:47:in `sample'
2016-11-10_17:16:11.01095 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:36:in `block (2 levels) in start'
2016-11-10_17:16:11.01095 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `loop'
2016-11-10_17:16:11.01096 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `block in start'
==> /var/log/gitlab/unicorn/unicorn_stderr.log <==
E, [2016-11-10T17:16:07.952528 #552] ERROR -- : /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:61:in `block in with'
E, [2016-11-10T17:16:07.952556 #552] ERROR -- : /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `handle_interrupt'
E, [2016-11-10T17:16:07.952582 #552] ERROR -- : /opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `with'
E, [2016-11-10T17:16:07.952609 #552] ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics.rb:44:in `submit_metrics'
E, [2016-11-10T17:16:07.952637 #552] ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:54:in `flush'
E, [2016-11-10T17:16:07.952670 #552] ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:47:in `sample'
E, [2016-11-10T17:16:07.952703 #552] ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:36:in `block (2 levels) in start'
E, [2016-11-10T17:16:07.952851 #552] ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `loop'
E, [2016-11-10T17:16:07.952887 #552] ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `block in start'
I, [2016-11-10T17:16:49.922143 #451] INFO -- : Refreshing Gem list
==> /var/log/gitlab/gitlab-rails/production.log <==
SocketError (getaddrinfo: Name or service not known):
lib/gitlab/metrics.rb:149:in `new'
lib/gitlab/metrics.rb:149:in `block in <module:Metrics>'
lib/gitlab/metrics.rb:44:in `submit_metrics'
lib/gitlab/metrics/transaction.rb:126:in `submit'
lib/gitlab/metrics/transaction.rb:98:in `finish'
lib/gitlab/metrics/rack_middleware.rb:44:in `call'
lib/gitlab/middleware/go.rb:16:in `call'
Output of checks
Results of GitLab application Check
root@gitlab:/# gitlab-rake gitlab:check SANITIZE=true
Checking GitLab Shell ...
GitLab Shell version >= 3.6.6 ? ... OK (3.6.6)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
54/1 ... ok
51/48 ... ok
54/98 ... ok
51/100 ... ok
51/102 ... ok
51/104 ... ok
51/107 ... ok
51/109 ... ok
52/110 ... ok
53/111 ... ok
55/119 ... ok
56/120 ... ok
53/125 ... ok
52/126 ... ok
55/128 ... ok
52/129 ... ok
55/130 ... ok
52/134 ... ok
52/136 ... ok
52/137 ... ok
55/138 ... ok
52/139 ... ok
52/140 ... ok
54/145 ... ok
52/146 ... ok
58/147 ... ok
52/148 ... ok
52/149 ... ok
52/150 ... ok
52/151 ... repository is empty
59/156 ... ok
59/157 ... ok
54/158 ... ok
54/159 ... ok
52/160 ... ok
51/163 ... ok
54/164 ... ok
52/166 ... ok
52/168 ... ok
52/169 ... ok
55/170 ... ok
59/171 ... repository is empty
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: FAILED. code: 500
gitlab-shell self-check failed
Try fixing it:
Make sure GitLab is running;
Check the gitlab-shell configuration file:
sudo -u git -H editor /opt/gitlab/embedded/service/gitlab-shell/config.yml
Please fix the error above and rerun the checks.
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured with autocrlf=input? ... yes
Database config exists? ... yes
All migrations up? ... rake aborted!
SocketError: getaddrinfo: Name or service not known
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/writer/udp.rb:13:in `connect'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/writer/udp.rb:13:in `initialize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/client.rb:64:in `new'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/influxdb-0.2.3/lib/influxdb/client.rb:64:in `initialize'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics.rb:149:in `new'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics.rb:149:in `block in <module:Metrics>'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:169:in `try_create'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:81:in `block (2 levels) in pop'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:77:in `loop'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:77:in `block in pop'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:76:in `synchronize'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool/timed_stack.rb:76:in `pop'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:89:in `checkout'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:61:in `block in with'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `handle_interrupt'
/opt/gitlab/embedded/service/gem/ruby/2.3.0/gems/connection_pool-2.2.0/lib/connection_pool.rb:60:in `with'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics.rb:44:in `submit_metrics'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:54:in `flush'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:47:in `sample'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:36:in `block (2 levels) in start'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `loop'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/sampler.rb:33:in `block in start'
Tasks: TOP => gitlab:check => gitlab:app:check
(See full trace by running task with --trace)
Results of GitLab environment info
root@gitlab:/# gitlab-rake gitlab:env:info
System information
System:
Current User: git
Using RVM: no
Ruby Version: 2.3.1p112
Gem Version: 2.6.6
Bundler Version:1.13.6
Rake Version: 10.5.0
Sidekiq Version:4.2.1
GitLab information
Version: 8.13.5
Revision: 09cedb5
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: postgresql
URL: https://gitlab.xxxxxx.xxx
HTTP Clone URL: https://gitlab.xxxxxx.xx/some-group/some-project.git
SSH Clone URL: git@gitlab.xxxxxx.xxx:some-group/some-project.git
Using LDAP: no
Using Omniauth: no
GitLab Shell
Version: 3.6.6
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks/
Git: /opt/gitlab/embedded/bin/git
Possible fixes
(If you can, link to the line of code that might be responsible for the problem)