Run KAS tests against GitLab deployed to Openshift with the Operator

Summary

As a follow-up from the discusion from #353 (comment 1302898400), there came the idea to run gitlab-org/gitlab QA tests against a GitLab instance managed by the Operator and deployed to Openshift.

Currently QA tests already run against Omnibus versions on every commit, and against the chart deployed to EKS and GKE, via chart MRs.

The discussion initially started by running the KAS QA spec against an Openshift installation. But this could actually be extended to any GitLab QA specs from the gitlab-qa framework.. We already run other QA tests against the Operator.

Known limitations

Steps to reproduce

I can confirm that, if we handle these two limitations, the tests can run successfully against a Openshift GitLab instance. How to run them:

  • Create a public group in you instance to use it as your GITLAB_SANDBOX_NAME.

  • Create a access token with the api scope for the same user who manages the group above.

  • Have your local gcloud CLI logged in and setup to access your GCP project sandbox, where you should have access to manage Kubernetes clusters.

  • Run the following (override the env variables by yours)

    GITLAB_QA_ACCESS_TOKEN="${YOUR_API_ACCESS_TOKEN}" \
GITLAB_SANDBOX_NAME="${YOUR_SANDBOX_GROUP_NAME}" \
GITLAB_USERNAME="${YOUR_USERNAME}" \
GITLAB_PASSWORD="${YOUR_PASSWORD}" \
bundle exec bin/qa Test::Instance::All https://${YOUR_GITLAB_INSTANCE_DOMAIN} -- -- qa/specs/features/ee/api/7_configure/kubernetes/kubernetes_agent_spec.rb

Proposals

For environments limitation

Since the KAS tests are only runnable to specific environments (staging, staging-canary). We need to make this test runnable also specifically for this project.

Both proposals need a bit more of code investigation to understand which is the best approach.

Proposal 1: Ignore subdomain restriction

Change GitLab QA code to be able to ignore subdomain restrictions and run specific test if it's being called with a metadata.

More details

Metadata is being recognized when you add a new one so the test can be picked up if you just add :custom_tag to it, then [Quarantine/context check](https://gitlab.com/gitlab-org/gitlab/-/blob/7359d23f4e078479969c872924150219c6f1665f/qa/qa/specs/helpers/quarantine.rb#L61 happens before test is being run. So with custom tag we can run the test directly but it still being skipped later on quarantine/context check. So perhaps we should add an option to ignore context restriction, but it may be involved. If there is no plan to move Operator CI to a different domain, it might be easier to go with proposal 2.

Proposal 2: Add Operator's Subdomain

Extend the Subdomain metadata to include the Subdomain that we use for the gitlab-operator CI tests, to selective enable this project.

Proposals for installing the agent

  • Add GCP credentials for creating GKE clusters.
  • Migrate the solution to use a K3d cluster instead. The K3d cluster needs access to have network access to the KAS address deployed to the CI cluster.
Edited by Nailia Iskhakova