Skip to content

LetsEncrypt renewal failures may require removal of /etc/acme/account_private_key.pem

The update to chef-acme from v4.1.3 to v4.1.4 in GitLab 14.6 (!5590 (merged)) introduced a significant behavioral change caused by https://github.com/schubergphilis/chef-acme/commit/3139ecee36f7094036b3e168a3e2e7edceca6672.

Prior to GitLab 14.6, the RSA private key was created in-memory every time and never persisted. Now, it is written to /etc/acme/account_private_key.pem and reused whenever possible.

I ran into this a week ago in https://gitlab.com/gitlab-org/container-registry/-/issues/535#note_792288038 because I had changed contact_emails, but ran into a problem where gitlab-ctl reconfigure would always fail with Acme::Client::Error::AccountDoesNotExist: No account exists with the provided key error:

    Acme::Client::Error::AccountDoesNotExist
    ----------------------------------------
    acme_certificate[staging] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/letsencrypt/resources/certificate.rb line 41) had an error: Acme::Client::Error::AccountDoesNotExist: No account exists with the provided key

    Cookbook Trace:
    ---------------
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/libraries/acme.rb:58:in `acme_order_certs_for'
    /opt/gitlab/embedded/cookbooks/cache/cookbooks/acme/resources/certificate.rb:89:in `block in class_from_file'

I think we should consider:

  1. Reverting back to v4.1.3 for 14.6.
  2. Prune /etc/acme/account_private_key.pem before every reconfigure run (?).
  3. Serialize the state of LetsEncrypt config and make it dependent on the hash of the private key. (?)

/cc: @twk3, @WarheadsSE

Edited by Stan Hu