Kubernetes CA rotation for ops-gitlab-gke

The existing Kubernetes CA in ops-gitlab-gke is going to expire on 2025-03-31:

❯ gcloud container clusters describe ops-gitlab-gke --project gitlab-ops --region us-east1 --format "value(masterAuth.clusterCaCertificate)" | base64 --decode | openssl x509 -noout -dates
notBefore=Apr  1 03:23:26 2020 GMT
notAfter=Mar 31 04:23:26 2025 GMT

This requires manual intervention to make sure we rotate the CA without impact/outage to the workloads and deployments.

Steps for rotation are available in the official GKE documentation.

Timing is important here, GKE automatically starts a CA rotation 30 days before it expires.

Validate

• Workload static ServiceAccount secrets/tokens
  • Switch them to short-lived tokens if possible, otherwise rotate them once the new CA is created
• Deployment/CI ServiceAccount tokens, rotate them once the new CA is created