Draft: Use vcluster for QA jobs

Jon Dovestonrequested to merge
ci-use-vcluster-for-qa into master

What does this MR do?

This MR adds full QA pipeline jobs that use vclusters. The plan is leave the current version testing jobs in place while this new vcluster approach is tested and optimised (stability, reliability, performance, scalability, etc).

The deploy and test logic in the existing jobs has been duplicated into scripts, and refactored adapted for vcluster. In the future perhaps these scripts could be backported to the non-vcluster jobs and simplified. But for now an attempt has been made to avoid changes to the existing jobs as much as possible. This approach also means the new jobs are independent of existing jobs, they do not extend or import any code from the existing jobs (with the exception of some rules).

  • The new jobs have been added to the vcluster downstream trigger along side the current vcluster smoke tests
  • The new jobs are named *_gkevcNNN.
  • They use the gkevc-ci-cluster agent that targets the new cloud-native-vcluster GKE cluster.
  • The existing vcluster smoke tests have been switched to this agent as well.
  • The new jobs are currently allow_failure: true.
  • The vcluster tool version has been upgraded (at the time of writing this version is installed at runtime and is not embedded in the container image)
  • The new deploys install into the default namespace in the vcluster.
  • The wild card TLS cert is copied to the vcluster.
  • External-dns in the host cluster creates records from nginx LoadBalancer service annotations not from the Ingress.
  • The stop jobs will delete the vcluster (and its namespace) in the host cluster.
  • The vcluster create script adds kube-janitor annotations with a 2 day TTL.

Related issues

Related to gitlab-org/distribution/team-tasks#1445 (closed)

Author checklist

For general guidance, please follow our Contributing guide.

Required

For anything in this list which will not be completed, please provide a reason in the MR discussion.

  • Merge Request Title and Description are up to date, accurate, and descriptive.
  • MR targeting the appropriate branch.
  • MR has a green pipeline.
  • Documentation created/updated.
  • Tests added/updated, and test plan for scenarios not covered by automated tests.
  • Equivalent MR/issue for omnibus-gitlab opened.

Reviewers checklist

Edited by Jon Doveston

Merge request reports