`gitlab-ctl renew-le-certs` won't renew an expired cert
Summary
gitlab-ctl renew-le-certs
won't renew an expired cert
Steps to reproduce
- Have a cert expired yesterday
- Try
gitlab-ctl renew-le-certs
to update the cert - Notice the cert hasn't changed
What is the current bug behavior?
Certificate is not updated
What is the expected correct behavior?
Certificate should be renewed
Relevant logs
Relevant logs
root@tpresa-01:~# gitlab-ctl renew-le-certs Starting Chef Client, version 13.6.4 resolving cookbooks for run list: ["gitlab::letsencrypt_renew"] Synchronizing Cookbooks: - gitlab (0.0.1) - package (0.1.0) - postgresql (0.1.0) - redis (0.1.0) - mattermost (0.1.0) - registry (0.1.0) - gitaly (0.1.0) - consul (0.1.0) - letsencrypt (0.1.0) - nginx (0.1.0) - runit (4.3.0) - acme (3.1.0) - crond (0.1.0) - compat_resource (12.19.1) Installing Cookbook Gems: Compiling Cookbooks... Converging 0 resourcesRunning handlers: Running handlers complete Chef Client finished, 0/0 resources updated in 07 seconds root@tpresa-01:~# gitlab-ctl renew-le-certs Starting Chef Client, version 13.6.4 resolving cookbooks for run list: ["gitlab::letsencrypt_renew"] Synchronizing Cookbooks:
- gitlab (0.0.1)
- package (0.1.0)
- postgresql (0.1.0)
- redis (0.1.0)
- registry (0.1.0)
- mattermost (0.1.0)
- consul (0.1.0)
- gitaly (0.1.0)
- letsencrypt (0.1.0)
- nginx (0.1.0)
- runit (4.3.0)
- crond (0.1.0)
- acme (3.1.0)
- compat_resource (12.19.1) Installing Cookbook Gems: Compiling Cookbooks... Converging 0 resources
Running handlers: Running handlers complete Chef Client finished, 0/0 resources updated in 13 seconds root@tpresa-01:~# openssl s_client -connect gitlab.tpresa.com:443 CONNECTED(00000003) depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3 verify return:1 depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3 verify return:1 depth=0 CN = gitlab.tpresa.com verify error:num=10:certificate has expired notAfter=Apr 3 17:04:58 2019 GMT verify return:1 depth=0 CN = gitlab.tpresa.com notAfter=Apr 3 17:04:58 2019 GMT verify return:1
Certificate chain 0 s:/CN=gitlab.tpresa.com i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 1 s:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3 i:/O=Digital Signature Trust Co./CN=DST Root CA X3
Server certificate -----BEGIN CERTIFICATE----- MIIFXDCCBESgAwIBAgISA+rkSL76I4+1ekjVRdgVoDRcMA0GCSqGSIb3DQEBCwUA MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xOTAxMDMxNzA0NThaFw0x OTA0MDMxNzA0NThaMBwxGjAYBgNVBAMTEWdpdGxhYi50cHJlc2EuY29tMIIBIjAN BgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAnod3jcGaxdgAG5DvS1MkyGakyTdS Lyx619RBT3Kbq/38l93N/mWmvGip3gMEuS0oC4276kvkgKPpGo8C3+WyqxQoMAcl wt7cu+Fm38MaY9ysQgok1BMC4cCnn53fCVzouNwBDk/3FBe9x6ceJNbME2UIubV0 rCwUiL2RJxKueyCjin73S6wdrnKDvV3eogzRsGMV2wtkEsWm4ivmvf2brLrw2g1v OTEx6SFv6Kee3r89itr3jV2d4cv4QvSNvg5s3QQf054XPXXV9np+xaDJf1jGU58Z OSi1OdDkmDCqanIVzFBrEg0fPyzHKzuMWuFc3tN+DHQ9NliHcJijFRuQMwIDAQAB o4ICaDCCAmQwDgYDVR0PAQH/BAQDAgWgMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggr BgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQWBBSN1R6P2x8OZC+FDDUdKDto Eb4pUTAfBgNVHSMEGDAWgBSoSmpjBH3duubRObemRWXv86jsoTBvBggrBgEFBQcB AQRjMGEwLgYIKwYBBQUHMAGGImh0dHA6Ly9vY3NwLmludC14My5sZXRzZW5jcnlw dC5vcmcwLwYIKwYBBQUHMAKGI2h0dHA6Ly9jZXJ0LmludC14My5sZXRzZW5jcnlw dC5vcmcvMBwGA1UdEQQVMBOCEWdpdGxhYi50cHJlc2EuY29tMEwGA1UdIARFMEMw CAYGZ4EMAQIBMDcGCysGAQQBgt8TAQEBMCgwJgYIKwYBBQUHAgEWGmh0dHA6Ly9j cHMubGV0c2VuY3J5cHQub3JnMIIBBgYKKwYBBAHWeQIEAgSB9wSB9ADyAHcAdH7a gzGtMxCRIZzOJU9CcMK//V5CIAjGNzV55hB7zFYAAAFoFOPG8QAABAMASDBGAiEA 7HXXs//XjoySO1mrRYcwkCuYQcHVbgnlcfuO+ZGCrlICIQDftNskyYxAwWuKcpbx 0ihx975nzZ89LJGtucs9tUs7IAB3ACk8UZZUyDlluqpQ/FgH1Ldvv1h6KXLcpMMM 9OVFR/R4AAABaBTjx1YAAAQDAEgwRgIhAJwYt1beiUpnxfGOFZWp+l+skXZjKbZL uFGIcHdtZ90UAiEA6pN1q2qOnlkY5gbAVYJ0+WUkiMSn56pzec0V00JX5RwwDQYJ KoZIhvcNAQELBQADggEBACBiS4JMkgajTqJusu6eO2oN4s9yYAfjS6fYE/rES1IC 7U7WcH2MdXm5/DkJm1KV4gOr1lmtX2cSjQrXbDw8VLQAfl1PexmzMoIRAFzt9sqi yrrgj2rc8mEDh9NLlktSL+8gdWgWVi+j1H+dkAVg+9nfpYh/kmwZYhtyStmHrpo9 ETM7SW4TCo6gOUjHIAj1Yagk+6mHGC0XVHnaUbqkN8b1DEQRhQzhXH5i5wmtp2SQ 5Qk2JaYznAsvgD6EOg759Jd//lsyn1GCxqC0A4UEqzvtvSTUmes+gcihhOSyUH1J Uh4fi+jaotKt4/goBZoMlMj4N14EPK4iePbcf0GQ6sg= -----END CERTIFICATE----- subject=/CN=gitlab.tpresa.com issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
No client certificate CA names sent Peer signing digest: SHA512 Server Temp Key: ECDH, P-256, 256 bits
SSL handshake has read 3223 bytes and written 302 bytes Verification error: certificate has expired
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: 2750309A4A830958A614F191F52C0EF85B85934FD3B7DEFEF628CB7AC7C5A7D3
Session-ID-ctx:
Master-Key: 1F5E4C4DF91EA56E1E093F6526A4A0A232C4467B17FA06070E0F522DD54FE5F42CFE4706E3650DB98CC5265B1886BA1A
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 300 (seconds)
TLS session ticket:
0000 - 27 99 c9 4b 0a 1a c5 d4-1b a6 a9 f4 96 58 ed fb '..K.........X..
0010 - 78 92 64 d8 92 4d 50 3e-66 5a f4 48 42 e7 f9 bf x.d..MP>fZ.HB...
0020 - e7 a3 7c 2a c6 d5 01 56-58 11 9e f6 74 f0 22 b7 ..|*...VX...t.".
0030 - 24 4e ac 85 fc 67 a6 8e-d6 ce 1e 60 5f 9d 1f 12 $N...g....._... 0040 - 50 a1 07 85 5e c0 24 e4-9c c3 46 9b 3b 4c 38 93 P...^.$...F.;L8. 0050 - c4 b9 69 0c cc 53 c5 67-16 a0 d9 99 2d e6 3a 74 ..i..S.g....-.:t 0060 - 6c 8a 1a c7 4f 5a 3b 4e-7f ef 08 30 b4 3e d2 60 l...OZ;N...0.>.
0070 - ff 06 63 62 c1 d8 12 2c-f5 aa d7 c7 f8 07 15 2d ..cb...,.......-
0080 - 6f 51 e1 c9 a8 df d3 26-c0 48 8c 2a c2 62 9d 33 oQ.....&.H.*.b.3
0090 - 84 b7 b6 eb 8f bd b4 39-6c 8f 41 a1 43 c4 a1 74 .......9l.A.C..t
00a0 - 27 46 22 1d 1f 52 9e 4e-8b ae 54 50 f6 b4 4f 3b 'F"..R.N..TP..O;
Start Time: 1554394897
Timeout : 7200 (sec)
Verify return code: 10 (certificate has expired)
Extended master secret: no
Details of package version
Provide the package version installation details
root@tpresa-01:~# dpkg-query -l "gitlab-*" Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) ||/ Name Version Architecture Description +++-======================================-========================-========================-================================================================================== un gitlab-ce (no description available) un gitlab-ci-multi-runner (no description available) un gitlab-ci-multi-runner-beta (no description available) ii gitlab-ee 11.9.4-ee.0 amd64 GitLab Enterprise Edition (including NGINX, Postgres, Redis) ii gitlab-runner 11.7.0 amd64 GitLab Runner un gitlab-runner-beta (no description available)
Environment details
- Operating System:
Ubuntu 18.04.1 LTS \n \l
- Installation Target, remove incorrect values:
- VM:
Digital Ocean
- VM:
- Installation Type, remove incorrect values:
- Upgrade from version
11.8
- Upgrade from version
- Is there any other software running on the machine:
gitlab runner, grafana
- Is this a single or multiple node installation?
- Resources
- CPU:
4
- Memory total:
8GB
- CPU:
Configuration details
Provide the relevant sections of `/etc/gitlab/gitlab.rb`
root@tpresa-01:~# grep -v -e '^#' -e '^$' /etc/gitlab/gitlab.rb external_url 'https://gitlab.tpresa.com' gitlab_rails['packages_enabled'] = true gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8', '::1/128'] gitlab_rails['omniauth_allow_single_sign_on'] = ['saml'] gitlab_rails['omniauth_block_auto_created_users'] = false gitlab_rails['omniauth_auto_link_saml_user'] = true gitlab_rails['omniauth_providers'] = [ { name: 'saml', args: { assertion_consumer_service_url: 'https://gitlab.tpresa.com/users/auth/saml/callback', idp_cert_fingerprint: '06:A0:6E:2E:AC:6C:2E:A2:52:B7:5B:D6:22:EF:36:1E:A5:D5:A1:C7', idp_sso_target_url: 'https://dev-435603.oktapreview.com/app/gitlabdev435603_tpresagitlab_1/exkiqvr81v6ppTFla0h7/sso/saml', issuer: 'https://gitlab.tpresa.com', name_identifier_format: 'urn:oasis:names:tc:SAML:2.0:nameid-format:transient' }, label: 'Okta' # optional label for SAML login button, defaults to "Saml" } ] grafana['enable'] = true registry_external_url 'https://gitlab.tpresa.com:4567' prometheus['enable'] = true prometheus['listen_address'] = '0.0.0.0:9090'