[CI] Move EKS cluster to new aws sandbox and use k8s 1.20 instead of 1.19
Update our EKS cluster to 1.20
Amazon is dropping support for 1.19 on June 30 - https://docs.aws.amazon.com/eks/latest/userguide/kubernetes-versions.html#kubernetes-release-calendar
Upgrade docs are here: https://docs.aws.amazon.com/eks/latest/userguide/update-cluster.html
And previous upgrade issue is here: #3020 (closed)
For the this upgrade we actually want to setup a new cluster in our new AWS sandbox: https://gitlab.com/gitlab-com/business-technology/engineering/infrastructure/issue-tracker/-/issues/275
Some things to note:
- We have been using eksctl to setup our EKS clusters
- A maintainer will need to attach the new cluster to the project via the deprecated certificate k8s method. This is because our CI jobs do some kubectl exec which are not supported on the gitlab agent through gitlab.com right now.
- The eks.helm-charts.win subdomain is managed in the the current top accounts route53, but pointed there from our GCP cloud native project. The subdomain will need to be moved to the new sandbox in order for externalDNS in the cluster to work in our jobs. We might want to play with using a new subdomain to confirm the transition works before cutting over like
eks-new.helm-charts.win
- certmanager in the cluster is setup with a wildcard cert.
Edited by DJ Mountney