Roll out renewed cert for *.staging.gitlab.io

Background

We got a blackbox SSLCertExpiresSoon alert for this. The cert expires on 2026-06-26. We should do it sooner than that, to avoid paging SRE on-call.

Analysis

We can see in https://sslmate.com/console/orders/ that the cert was renewed, new cert is expiring 2026-05-17.

But we can see that this cert has not propagated to pages:

➜  ~ echo | openssl s_client -connect tls-test.staging.gitlab.io:443 | openssl x509 -text -noout | grep -E 'Subject:|Not Before:|Not After :'

            Not Before: May 27 00:00:00 2024 GMT
            Not After : Jun 25 23:59:59 2025 GMT
        Subject: CN=*.staging.gitlab.io

➜  ~ glsh kube use-cluster gstg-us-east1-b
➜  ~ k get secrets -n gitlab gitlab-pages-tls-certificate-v5 -o json | jq -r '.data."tls.crt"|@base64d' | openssl x509 -text -noout | grep -E 'Subject:|Not Before:|Not After :'

            Not Before: May 27 00:00:00 2024 GMT
            Not After : Jun 25 23:59:59 2025 GMT
        Subject: CN=*.staging.gitlab.io

According to external-secrets, the path in vault is k8s/env/gstg/ns/gitlab/pages/tls (link):

➜  ~ k describe externalsecrets -n gitlab gitlab-pages-tls-certificate-v5

      Key:                  env/gstg/ns/gitlab/pages/tls
      Metadata Policy:      None
      Property:             certificate
      Version:              5

This comment from @cmcfarland suggests that the process of updating the cert in vault is not handled by certificates-updater.

Steps

1. Download cert from sslmate
2. Update cert in vault
3. Create new secret version in k8-workloads/gitlab-com
4. Point deployment to new secret version
5. Wait for propagation
6. Verify successful rollout (see below)
7. Remove old version from k8-workloads/gitlab-com

Verification

➜  ~ echo | openssl s_client -connect tls-test.staging.gitlab.io:443 | openssl x509 -text -noout | grep -E 'Subject:|Not Before:|Not After :'