LetsEncrypt renewal failures may require removal of /etc/acme/account_private_key.pem
The update to chef-acme from v4.1.3 to v4.1.4 in GitLab 14.6 (!5590 (merged)) introduced a significant behavioral change caused by https://github.com/schubergphilis/chef-acme/commit/3139ecee36f7094036b3e168a3e2e7edceca6672.
Prior to GitLab 14.6, the RSA private key was created in-memory every time and never persisted. Now, it is written to /etc/acme/account_private_key.pem
and reused whenever possible.
I ran into this a week ago in https://gitlab.com/gitlab-org/container-registry/-/issues/535#note_792288038 because I had changed contact_emails
, but ran into a problem where gitlab-ctl reconfigure
would always fail with Acme::Client::Error::AccountDoesNotExist: No account exists with the provided key
error:
Acme::Client::Error::AccountDoesNotExist
----------------------------------------
acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 41) had an error: Acme::Client::Error::AccountDoesNotExist: No account exists with the provided key
Cookbook Trace:
---------------
/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/libraries/acme.rb:58:in `acme_order_certs_for'
/opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:89:in `block in class_from_file'
I think we should consider:
- Reverting back to v4.1.3 for 14.6.
- Prune
/etc/acme/account_private_key.pem
before everyreconfigure
run (?). - Serialize the state of LetsEncrypt config and make it dependent on the hash of the private key. (?)
/cc: @twk3, @WarheadsSE