Skip to content

CI: Replace 'pet' clusters with ephemeral clusters

Summary

If we find a solution to #1445 (closed), let's try to replace our 'pet' clusters with 'cattle' clusters via vcluster.

We're already using vcluster to create ephemeral environments for smoke testing the GitLab Charts against upcoming versions of Kubernetes. If we can find a way to run QA against instances running in vcluster, we should be able to remove our 'pet' clusters entirely. At that point, we can create one large cluster and let it auto-update to the latest version, using vcluster to pin us to specific versions of Kubernetes we want to support.

Some benefits include:

  • No longer having to provision clusters for each new version of Kubernetes
  • No longer having cloud providers auto-update our clusters and cause mismatches between the version referenced in the job and the version running in the cluster
  • We should end up with much more predictable, consistent, deterministic Review App environments that require less maintenance toil
  • Bonus: if we can find a way in #1445 (closed) to 'mock' a LoadBalancer (MetalLB?) we could end up with significant cost savings by not needing our cloud provider to provision one for every Helm Chart instance.
  • Others?

Acceptance criteria

  • vcluster instances are used for QA testing in Merge Request and master/stable pipelines
  • 'Pet' clusters are deleted, leaving one per project as a foundation/host for vcluster
Edited by Mitchell Nielsen