CI: Replace 'pet' clusters with ephemeral clusters
Summary
If we find a solution to #1445, let's try to replace our 'pet' clusters with 'cattle' clusters via vcluster
.
We're already using vcluster
to create ephemeral environments for smoke testing the GitLab Charts against upcoming versions of Kubernetes. If we can find a way to run QA against instances running in vcluster
, we should be able to remove our 'pet' clusters entirely. At that point, we can create one large cluster and let it auto-update to the latest version, using vcluster
to pin us to specific versions of Kubernetes we want to support.
Some benefits include:
- No longer having to provision clusters for each new version of Kubernetes
- No longer having cloud providers auto-update our clusters and cause mismatches between the version referenced in the job and the version running in the cluster
- We should end up with much more predictable, consistent, deterministic Review App environments that require less maintenance toil
- Bonus: if we can find a way in #1445 to 'mock' a LoadBalancer (MetalLB?) we could end up with significant cost savings by not needing our cloud provider to provision one for every Helm Chart instance.
- Others?
Acceptance criteria
-
vcluster
instances are used for QA testing in Merge Request andmaster
/stable
pipelines -
'Pet' clusters are deleted, leaving one per project as a foundation/host for vcluster
Edited by Mitchell Nielsen