Skip to content

OpenIDConnect::Discovery::DiscoveryFailed in Gitlab Version 14.X

Summary

Since upgrade to Gitlab version 14.X (starting with chart version 5.0.5) our Keycloak certificate is no longer trusted by the system and SSO logins are showing HTTP/500 errors. We are using global.certificates.customCAs to feed in the root CA for the IdP certificate which was successful in gitlab 13.X versions.

Steps to reproduce

Enable and configure SSO.

Create secret with root CA in gitlab namespace, point to secret under global.certificates.customCAs

Attempt Login via SSO to Gitlab

Configuration used

(Please provide a sanitized version of the configuration used wrapped in a code block (```yaml))

global:
  appConfig:
    omniauth:
      enabled: true
      allowSingleSignOn: ['openid_connect']
      blockAutoCreatedUsers: false
      providers:
        - secret: gitlab-sso-provider
          key: gitlab-sso.json
  certificates:
    customCAs:
      - secret: tls-ca-sso
gitlab-sso.json: {
  "name": "openid_connect",
  "label": "Dev Keycloak",
  "args": {
    "name": "openid_connect",
    "scope": [
      "Gitlab"
    ],
    "response_type": "code",
    "issuer": "https://keycloak.example.com/auth/realms/omni",
    "client_auth_method": "query",
    "discovery": true,
    "uid_field": "preferred_username",
    "client_options": {
      "identifier": "gitlab",
      "secret": "XXXXXXXXXXXXXX",
      "redirect_uri": "https://gitlab.example.com/users/auth/openid_connect/callback",
      "end_session_endpoint": "https://keycloak.example.com/auth/realms/omni/protocol/openid-connect/logout"
    }
  }
}

Current behavior

Gitlab UI is showing 500 errors when attempting to use openidconnect SSO and relevant output in the logs:

OpenIDConnect::Discovery::DiscoveryFailed (SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate in certificate chain)):

Expected behavior

Successful redirect to Keycloak & back and showing Gitlab as logged in user.

Versions

  • Chart: 5.0.5 & 5.3.1
  • Platform:
    • Self-hosted: K3D & RKE2
  • Kubernetes: (kubectl version)
    • Client: v1.19.0
    • Server: v1.21.2+k3s1
  • Helm: (helm version)
    • Client: v3.5.4
    • Server: fluxcd/helm-controller v0.11.0

Relevant logs

==> /var/log/gitlab/production.log <==
Started POST "/users/auth/openid_connect" for 10.42.1.5 at 2021-12-07 17:47:36 +0000
Processing by Gitlab::RequestForgeryProtection::Controller#index as HTML
  Parameters: {"authenticity_token"=>"[FILTERED]"}
Completed 200 OK in 1ms (ActiveRecord: 0.0ms | Elasticsearch: 0.0ms | Allocations: 156)

OpenIDConnect::Discovery::DiscoveryFailed (SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate
in certificate chain)):

lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'
lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'
lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'
lib/gitlab/metrics/web_transaction.rb:21:in `run'
lib/gitlab/metrics/rack_middleware.rb:16:in `call'
lib/gitlab/middleware/speedscope.rb:13:in `call'
lib/gitlab/request_profiler/middleware.rb:17:in `call'
lib/gitlab/jira/middleware.rb:19:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:21:in `call'
lib/gitlab/middleware/multipart.rb:172:in `call'
lib/gitlab/middleware/read_only/controller.rb:50:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/same_site_cookies.rb:27:in `call'
lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'
lib/gitlab/middleware/request_context.rb:21:in `call'
config/initializers/fix_local_cache_middleware.rb:11:in `call'
lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:74:in `call'
lib/gitlab/middleware/release_env.rb:12:in `call'
Started GET "/-/readiness" for 10.42.1.1 at 2021-12-07 17:47:44 +0000
Processing by HealthController#readiness as */*
Completed 200 OK in 1ms (Views: 0.3ms | ActiveRecord: 0.0ms | Elasticsearch: 0.0ms | Allocations: 234)
Edited by Ryan Garcia