LDAP Connection (OpenLDAP) no longer works: access denied for all accounts
Summary
I've upgraded from Gitlab 10.0 to gitlab-ce-10.5.3-ce.0.el7.x86_64 via the Omnibus packages. After the upgrade, all LDAP accounts are blocked and do not work anymore. Every user gets an access denied.
The error message in /gitlab-rails/aaplication.log is: March 06, 2018 14:05: LDAP account "cn=dkreyenb,ou=people,dc=foobar" does not exist anymore, blocking Gitlab user "Dirk Kreyenberg" (dirk.kreyenberg@gmail.com)
The exact same config worked in 10.0. Also the ldap server has been not touched.
Steps to reproduce
Log in with valid credientials result in "Access denied for you LDAP account" March 06, 2018 14:05: LDAP account "cn=dkreyenb,ou=people,dc=foobar" does not exist anymore, blocking Gitlab user "Dirk Kreyenberg" (dirk.kreyenberg@gmail.com)
When I use a bogus password the GUI tells me: March 06, 2018 14:05: Could not authenticate you from Ldapmain because "Invalid credentials for dkreyenb". This makes sense, apparently Gitlab gets something from the ldap server.
I also unblocked the user manually via the rails console, but after a new login try, it gets blocked again:
gitlab-rails console Loading production environment (Rails 4.2.10) irb(main):001:0> user = User.find_by_email("dirk.kreyenberg@gmail.com") => #<User id:2 @dkreyenb> irb(main):002:0> user.state="active" => "active" irb(main):003:0> user.save => true irb(main):004:0>
Example config
gitlab_rails['ldap_enabled'] = true
gitlab_rails['ldap_servers'] = { 'main' => { 'label' => 'LDAP', 'host' => 'x.x.x.x', 'port' => 636, 'uid' => 'uid', 'encryption' => 'simple_tls', 'verify_certificates' => true, 'bind_dn' => 'CN=ldapagent,OU=foobar,DC=foo,DC=bar', 'password' => '*******', 'active_directory' => false, 'base' => 'OU=people,DC=foobar', 'allow_username_or_email_login' => false, 'block_auto_created_users' => true, 'ca_file' => '/etc/openldap/cacerts/thecacert.pem' } }
What is the current bug behavior?
All ldap users cannot be authenticated anymore via ldap.
What is the expected correct behavior?
The ldap login should work and users shouldn't be blocked
Relevant logs and/or screenshots
production.log Started POST "/gitlab/users/auth/ldapmain/callback" for 10.92.39.25 at 2018-03-06 14:16:17 +0100 Processing by OmniauthCallbacksController#ldapmain as HTML Parameters: {"utf8"=>"✓", "authenticity_token"=>"[FILTERED]", "username"=>"dkreyenb", "password"=>"[FILTERED]"} Redirected to https://foo.bar/gitlab/users/sign_in Completed 302 Found in 254ms (ActiveRecord: 23.0ms)
application.log March 06, 2018 14:16: LDAP account "cn=dkreyenb,ou=people,dc=foobar" does not exist anymore, blocking Gitlab user "Dirk Kreyenberg" (dirk.kreyenberg@gmail.com)
Output of checks
This happens on a local gitlab-ce installation, no users were listed below.
gitlab-rake gitlab:ldap:check --trace ** Invoke gitlab:ldap:check (first_time) ** Invoke gitlab_environment (first_time) ** Invoke environment (first_time) ** Execute environment ** Execute gitlab_environment ** Execute gitlab:ldap:check Checking LDAP ...
Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results)
Checking LDAP ... Finished
Results of GitLab environment info
Expand for output related to GitLab environment info
System information System: Current User: git Using RVM: no Ruby Version: 2.3.6p384 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.3.0 Redis Version: 3.2.11 Git Version: 2.14.3 Sidekiq Version:5.0.5 Go Version: unknown
GitLab information Version: 10.5.3 Revision: ec4ac77 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://foo.bar/gitlab HTTP Clone URL: https://foo.bar/gitlab/some-group/some-project.git SSH Clone URL: git@foo.bar:some-group/some-project.git Using LDAP: yes Using Omniauth: no
GitLab Shell Version: 6.0.3 Repository storage paths:
- default: /ado/sw/gitlab-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
gitlab-rake gitlab:check SANITIZE=true Checking GitLab Shell ...
GitLab Shell version >= 6.0.3 ? ... OK (6.0.3) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 3/1 ... ok 10/2 ... ok 12/3 ... ok 11/4 ... ok 10/5 ... ok 3/6 ... ok 15/7 ... ok 15/8 ... repository is empty Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Reply by email is disabled in config/gitlab.yml Checking LDAP ...
Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results)
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 3/1 ... yes 10/2 ... yes 12/3 ... yes 11/4 ... yes 10/5 ... yes 3/6 ... yes 15/7 ... yes 15/8 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.3.6) Git version >= 2.9.5 ? ... yes (2.14.3) Git user has default SSH configuration? ... yes Active users: ... 5
Checking GitLab ... Finished
Possible fixes
No idea, is there is way to check an ldap connect in the "gitlab-rails console" and get a more verbose output?