Unable to renew letsencrypt certificate anymore (#4900) · Issues · GitLab.org / omnibus-gitlab

Unable to renew letsencrypt certificate anymore

### Summary I can't renew letsencrypt certificate anymore while it was working fine for several months. I'm running Omnibus, starter edition. ### Steps to reproduce I'm running omnibus on my own company server, I was using GitLab 12.5.0 and I wanted to upgrade to 12.5.3 when I figured out this issue. As suggested by the error message during the upgrade attempt, I ran `sudo gitlab-ctl reconfigure` to try to fix the issue but then I had this letsencrypt error message. ### Example Project Not sure I can provide anything to help here... ### What is the current *bug* behavior? When running `gitlab-ctl reconfigure` or `gitlab-ctl renew-le-certs`, I'm having the following error: ```shell Acme::Client::Error::Malformed ------------------------------ acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 25) had an error: Acme::Client::Error::Malformed: Method not allowed ``` The full log is attached: [sudo_gitlab-ctl_renew-le-certs.log](/uploads/927de42da1c16b91ed50884b5b701122/sudo_gitlab-ctl_renew-le-certs.log) **Please note that:** * the tcp port 80 and 443 are open to the internet (tested with telnet) * I have tried to shutdown the firewall of the host running gitlab: no change * this worked fine during several months * I've run through a lot of similar issues but no one had the exact same error message and all the working fixes (mainly about not open ports and wrong gitlab.rb configuration...) didn't work on my side * my gitlab.rb contains the following: ```ruby # nginx['enable'] = true # nginx['client_max_body_size'] = '250m' nginx['redirect_http_to_https'] = true nginx['redirect_http_to_https_port'] = 80 ... letsencrypt['enable'] = true # letsencrypt['contact_emails'] = # This should be an array of email addresses to add as contacts # letsencrypt['group'] = 'root' # letsencrypt['key_size'] = 2048 # letsencrypt['owner'] = 'root' # letsencrypt['wwwroot'] = '/var/opt/gitlab/nginx/www' # See http://docs.gitlab.com/omnibus/settings/ssl.html#automatic-renewal for more on these sesttings letsencrypt['auto_renew'] = true letsencrypt['auto_renew_hour'] = "2" # letsencrypt['auto_renew_minute'] = nil # Should be a number or cron expression, if specified. letsencrypt['auto_renew_day_of_month'] = "*/20" ``` ### What is the expected *correct* behavior? `gitlab-ctl reconfigure` should run successfully... ### Relevant logs and/or screenshots see above. ### Output of checks (If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com) #### Results of GitLab environment info <details> <summary>Expand for output related to GitLab environment info</summary> <pre> System information System: Proxy: no Current User: git Using RVM: no Ruby Version: 2.6.3p62 Gem Version: 2.7.9 Bundler Version:1.17.3 Rake Version: 12.3.3 Redis Version: 3.2.12 Git Version: 2.22.0 Sidekiq Version:5.2.7 Go Version: unknown GitLab information Version: 12.5.3-ee Revision: 63955893276 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 10.9 URL: https://gitlab-server-name.company.com HTTP Clone URL: https://gitlab-server-name.company.com/some-group/some-project.git SSH Clone URL: git@gitlab-server-name.company.com:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: yes Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 10.2.0 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git </pre> </details> #### Results of GitLab application Check <details> <summary>Expand for output related to the GitLab application check</summary> <pre> Checking GitLab subtasks ... Checking GitLab Shell ... GitLab Shell: ... GitLab Shell version >= 10.2.0 ? ... OK (10.2.0) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful Checking GitLab Shell ... Finished Checking Gitaly ... Gitaly: ... default ... OK Checking Gitaly ... Finished Checking Sidekiq ... Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1 Checking Sidekiq ... Finished Checking Incoming Email ... Incoming Email: ... Reply by email is disabled in config/gitlab.yml Checking Incoming Email ... Finished Checking LDAP ... LDAP: ... Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) User output sanitized. Found 7 users of 100 limit. Checking LDAP ... Finished Checking GitLab App ... Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 19/1 ... yes 3/4 ... yes 3/5 ... yes 7/10 ... yes 3/11 ... yes 3/13 ... yes 3/14 ... yes 3/15 ... yes 2/17 ... yes 13/19 ... yes 3/21 ... yes 17/23 ... yes 2/24 ... yes 19/25 ... yes 19/26 ... yes 19/28 ... yes 19/29 ... yes 21/30 ... yes 21/31 ... yes 21/32 ... yes 21/33 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.5.3 ? ... yes (2.6.3) Git version >= 2.22.0 ? ... yes (2.22.0) Git user has default SSH configuration? ... yes Active users: ... 8 Is authorized keys file accessible? ... yes Elasticsearch version 5.6 - 6.x? ... skipped (elasticsearch is disabled) Checking GitLab App ... Finished Checking GitLab subtasks ... Finished </pre> </details> ### Possible fixes (If you can, link to the line of code that might be responsible for the problem)

issue