LDAP SSL handshake failure since upgrade to 9.4.2
Summary
After updating Gitlab to 9.4.2, attempting to log in over LDAP results in the following error;
Could not authenticate you from Ldapmain because "Ssl connect returned=1 errno=0 state=sslv2/v3 read server hello a: sslv3 alert handshake failure".
Steps to reproduce
Upgrade to 9.4.2
What is the current bug behavior?
Logging in with LDAP fails with a SSL handshake failure.
What is the expected correct behavior?
Logging in with LDAP succeeds.
Relevant logs and/or screenshots
Attempting to log in results in;
==> /var/log/gitlab/gitlab-rails/production.log <==
Started POST "/users/auth/ldapmain/callback" for 130.236.16.159 at 2017-08-01 12:12:27 +0200
==> /var/log/gitlab/unicorn/unicorn_stdout.log <==
I, [2017-08-01T12:12:27.304191 #9319] INFO -- omniauth: (ldapmain) Callback phase initiated.
E, [2017-08-01T12:12:27.313736 #9319] ERROR -- omniauth: (ldapmain) Authentication failure! ldap_error: Net::LDAP::Error, SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: sslv3 alert handshake failure
==> /var/log/gitlab/gitlab-rails/production.log <==
Processing by OmniauthCallbacksController#failure as HTML
Parameters: {"utf8"=>"✓", "authenticity_token"=>"j3D0ROKvyaA+URAm/hWrfbmisC1hGo1NaiGQE56qQU2BC96zaNcEyLMC8s8mt/F4fbZ4ICiFxTMuaoVcVYirlQ==", "username"=>"aleol57", "password"=>"[FILTERED]"}
Redirected to https://lusitania.it.liu.se/users/sign_in
Completed 302 Found in 12ms (ActiveRecord: 0.0ms)
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
# gitlab-rake gitlab:env:infoSystem information System: CentOS 7.3.1611 Current User: git Using RVM: no Ruby Version: 2.3.3p222 Gem Version: 2.6.6 Bundler Version:1.13.7 Rake Version: 10.5.0 Redis Version: 3.2.5 Git Version: 2.13.0 Sidekiq Version:5.0.0 Go Version: unknown
GitLab information Version: 9.4.2 Revision: a2ffceb Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://gitlab.fqdn HTTP Clone URL: https://gitlab.fqdn/some-group/some-project.git SSH Clone URL: ssh://git@gitlab.fqdn:29418/some-group/some-project.git Using LDAP: yes Using Omniauth: yes Omniauth Providers: saml
GitLab Shell Version: 5.3.1 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
# gitlab-rake gitlab:check SANITIZE=true Checking GitLab Shell ...GitLab Shell version >= 5.3.1 ? ... OK (5.3.1) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 3/2 ... ok 10/3 ... ok 31/4 ... ok 5/5 ... ok 6/6 ... ok 6/7 ... ok 7/8 ... ok 6/9 ... ok 6/10 ... ok 6/11 ... ok 6/12 ... ok 6/13 ... ok 6/14 ... ok 6/17 ... ok 6/18 ... ok 6/19 ... ok 6/20 ... ok 6/21 ... ok 6/22 ... ok 6/23 ... ok 6/24 ... ok 6/25 ... ok 6/26 ... ok 6/27 ... ok 6/28 ... ok 6/29 ... ok 6/30 ... ok 6/31 ... ok 6/32 ... ok 6/33 ... ok 6/34 ... ok 6/36 ... ok 6/37 ... ok 6/38 ... ok 6/40 ... ok 6/49 ... ok 10/52 ... ok 6/58 ... ok 3/59 ... ok 6/60 ... ok 8/61 ... ok 9/63 ... ok 6/65 ... ok 3/69 ... ok 13/70 ... ok 13/71 ... ok 6/82 ... ok 3/83 ... ok 10/84 ... ok 12/85 ... ok 2/86 ... ok 9/87 ... ok 10/88 ... ok 6/89 ... ok 6/90 ... ok 6/91 ... ok 6/92 ... ok 6/93 ... ok 6/94 ... ok 6/96 ... ok 6/97 ... ok 6/98 ... ok 6/99 ... ok 6/100 ... ok 6/101 ... ok 3/102 ... ok 6/103 ... ok 6/104 ... ok 16/106 ... ok 3/108 ... ok 16/119 ... ok 16/120 ... ok 16/121 ... ok 16/122 ... ok 16/123 ... ok 16/124 ... ok 16/125 ... ok 3/127 ... ok 3/128 ... ok 3/130 ... ok 6/131 ... ok 7/132 ... ok 10/133 ... ok 9/134 ... ok 9/135 ... ok 5/136 ... repository is empty 5/137 ... repository is empty 14/139 ... ok 14/140 ... ok 14/141 ... ok 14/142 ... ok 14/144 ... ok 3/146 ... ok 14/147 ... ok 21/149 ... ok 6/150 ... ok 11/151 ... ok 3/152 ... ok 6/154 ... ok 7/163 ... ok 7/166 ... ok 2/167 ... ok 57/171 ... ok 3/177 ... ok 3/180 ... ok 20/182 ... ok 3/183 ... ok 7/184 ... ok 11/185 ... ok 3/187 ... ok 7/188 ... ok 4/189 ... ok 6/197 ... ok 6/198 ... ok 14/201 ... ok 6/202 ... ok 14/204 ... ok 24/205 ... ok 22/206 ... ok 24/207 ... ok 24/208 ... ok 16/210 ... ok 7/212 ... ok 16/213 ... ok 9/214 ... ok 6/215 ... ok 6/216 ... ok 14/217 ... ok 23/218 ... ok 12/219 ... ok 4/221 ... ok 29/222 ... ok 29/223 ... ok 2/225 ... ok 6/228 ... ok 2/229 ... ok 6/230 ... ok 6/232 ... ok 3/233 ... ok 2/234 ... ok 3/235 ... ok 14/237 ... ok 6/238 ... ok 6/239 ... ok 6/240 ... ok 6/241 ... ok 6/242 ... ok 6/243 ... ok 6/244 ... ok 6/245 ... ok 6/246 ... ok 6/247 ... ok 6/248 ... ok 6/249 ... ok 6/250 ... ok 6/251 ... ok 6/252 ... ok 6/253 ... ok 6/254 ... ok 6/255 ... ok 3/256 ... ok 2/257 ... ok 34/258 ... ok 3/259 ... ok 23/260 ... ok 4/261 ... ok 2/263 ... ok 3/264 ... ok 31/265 ... ok 34/266 ... ok 34/267 ... ok 3/271 ... ok 3/272 ... ok 6/273 ... ok 6/274 ... ok 3/275 ... ok 34/276 ... ok 3/278 ... ok 9/279 ... ok 11/280 ... ok 12/281 ... ok 7/283 ... ok 6/287 ... ok 11/288 ... ok 31/289 ... ok 24/290 ... ok 31/291 ... ok 31/292 ... ok 31/293 ... ok 9/294 ... ok 34/295 ... ok 3/296 ... ok 4/298 ... ok 4/299 ... ok 4/300 ... ok 34/301 ... ok 31/302 ... ok 38/303 ... ok 4/304 ... ok 24/305 ... ok 13/306 ... ok 14/307 ... ok 6/308 ... ok 6/313 ... ok 6/314 ... ok 6/315 ... ok 6/316 ... ok 6/317 ... ok 6/318 ... ok 6/319 ... ok 6/320 ... ok 6/321 ... ok 6/322 ... ok 6/323 ... ok 6/324 ... ok 6/325 ... ok 6/326 ... ok 6/327 ... ok 6/328 ... ok 6/329 ... ok 6/330 ... ok 6/331 ... ok 6/332 ... ok 6/333 ... ok 6/334 ... ok 6/335 ... ok 6/336 ... ok 6/337 ... ok 6/338 ... ok 6/339 ... ok 6/340 ... ok 6/341 ... ok 6/342 ... ok 34/343 ... ok 14/344 ... ok 11/347 ... ok 6/348 ... ok 7/349 ... ok 14/351 ... ok 2/352 ... ok 31/353 ... ok 38/354 ... ok 6/355 ... ok 38/356 ... ok 7/357 ... ok 14/359 ... ok 31/361 ... ok 24/362 ... ok 23/366 ... ok 17/367 ... ok 23/369 ... ok 14/372 ... ok 41/373 ... ok 2/374 ... ok 5/375 ... ok 4/376 ... ok 16/377 ... ok 17/378 ... ok 6/380 ... ok 6/381 ... ok 6/383 ... ok 6/384 ... ok 3/385 ... ok 17/386 ... ok 4/387 ... ok 6/389 ... ok 11/390 ... ok 7/391 ... ok 34/392 ... ok 3/393 ... repository is empty 2/394 ... ok 22/395 ... ok 41/396 ... ok 34/399 ... ok 4/400 ... ok 29/402 ... ok 31/404 ... ok 21/405 ... ok 7/407 ... ok 21/408 ... ok 3/409 ... ok 6/410 ... ok 6/412 ... ok 7/413 ... ok 55/415 ... ok 55/416 ... ok 55/417 ... ok 9/418 ... ok 2/419 ... ok 7/420 ... ok 6/421 ... ok 7/422 ... ok 38/423 ... ok 16/424 ... ok 7/425 ... ok 29/426 ... ok 31/427 ... ok 7/428 ... ok 31/429 ... ok 5/430 ... ok 3/431 ... ok 6/433 ... ok 3/434 ... ok 7/435 ... ok 5/437 ... ok 3/438 ... ok 31/439 ... ok 31/440 ... ok 4/441 ... ok 11/442 ... ok 16/443 ... ok 2/444 ... ok 4/445 ... ok 7/446 ... ok 11/447 ... ok 7/449 ... ok 7/450 ... ok 16/451 ... ok 7/452 ... ok 31/456 ... ok 14/457 ... ok 6/458 ... ok 6/459 ... ok 14/460 ... ok 14/461 ... ok 8/463 ... ok 14/464 ... ok 14/465 ... ok 6/466 ... ok 4/467 ... ok 11/468 ... ok 6/469 ... ok 17/470 ... ok 17/471 ... ok 58/472 ... ok 17/473 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Access to /var/opt/gitlab/.ssh/authorized_keys: OK Send ping to redis server: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Reply by email ...
Reply by email is disabled in config/gitlab.yml
Checking Reply by email ... Finished
Checking LDAP ...
Server: ldapmain rake aborted! Net::LDAP::Error: SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: sslv3 alert handshake failure /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/ldap/adapter.rb:7:in
open' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:472:in
block in check_ldap' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:468:ineach' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:468:in
check_ldap' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/check.rake:457:inblock (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:22:in
load' /opt/gitlab/embedded/bin/bundle:22:in `' Tasks: TOP => gitlab:check => gitlab:ldap:check (See full trace by running task with --trace)
Possible fixes
During my debugging, I discovered that it only seems to occur when specifying tls options.
irb(main):001:0> opts = c.adapter_options
=> {:host=>"ldap.master.fqdn", :port=>636, :encryption=>{:method=>:simple_tls, :tls_options=>{:verify_mode=>0}}}
irb(main):002:0> Net::LDAP.open(opts) do |ldap| ldap.search() end
Net::LDAP::Error: SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: sslv3 alert handshake failure
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/net-ldap-0.16.0/lib/net/ldap/connection.rb:72:in `open_connection'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/net-ldap-0.16.0/lib/net/ldap/connection.rb:698:in `socket'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/net-ldap-0.16.0/lib/net/ldap.rb:1321:in `new_connection'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/net-ldap-0.16.0/lib/net/ldap.rb:713:in `block in open'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/net-ldap-0.16.0/lib/net/ldap/instrumentation.rb:19:in `instrument'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/net-ldap-0.16.0/lib/net/ldap.rb:711:in `open'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/net-ldap-0.16.0/lib/net/ldap.rb:644:in `open'
from (irb):38
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.8/lib/rails/commands/console.rb:110:in `start'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.8/lib/rails/commands/console.rb:9:in `start'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.8/lib/rails/commands/commands_tasks.rb:68:in `console'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.8/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/railties-4.2.8/lib/rails/commands.rb:17:in `<top (required)>'
from bin/rails:9:in `require'
from bin/rails:9:in `<main>'
irb(main):003:0> opts[:encryption][:tls_options] = {}
=> {}
irb(main):004:0> opts
=> {:host=>"ldap.master.fqdn", :port=>636, :encryption=>{:method=>:simple_tls, :tls_options=>{}}}
irb(main):005:0> Net::LDAP.open(opts) do |ldap| ldap.search() end
=> nil
Changing lib/gitlab/ldap/config.rb
with the following patch allows gitlab:ldap:check
to succeed, though logins still seem to be broken;
--- a/config.rb 2017-08-01 12:01:40.764099971 +0200
+++ b/config.rb 2017-08-01 12:01:36.228943889 +0200
@@ -182,6 +182,7 @@
end
def tls_options(method)
+ return {}
return { verify_mode: OpenSSL::SSL::VERIFY_NONE } unless method
opts = if options['verify_certificates']
Downgraded the production GitLab to 9.4.1 for now, the issue doesn't appear on that version.