GitLab.com certificate not updated for HAProxy in production
See https://gitlab.com/gitlab-com/gl-infra/certificates-updater/-/jobs/6737402683
I, [2024-04-29T10:03:36.586246 #40] INFO -- : env/gprd/cookbook/gitlab-haproxy/frontend-loadbalancer / gitlab-haproxy.ssl.gitlab_crt :: certificate for gitlab.com is close to expiry (expires on 2024-05-11 23:59:59 UTC). Renewing.
E, [2024-04-29T10:03:36.707370 #40] ERROR -- : env/gprd/cookbook/gitlab-haproxy/frontend-loadbalancer / gitlab-haproxy.ssl.gitlab_crt :: matching private key with gitlab.com failed! Skipping ...
The key stored in the secret doesn't match the new certificate, it was likely rekeyed at some time between now and the last certificate update, same issue as for staging in #25306 (closed).
We need to update the new private key in the cookbook secret, can be copied from the secret for the Cloudflare certificate which does match.