Skip to content

feat(testing): Introduce e2e testing framework for Gitlab Observability Platform

Ankit Bhatnagar requested to merge abhatnagar/e2e-framework into main

Related to: #1830 (closed), #1723 (closed), #1722 (closed)

This commit introduces a new framework to allow for running E2E tests against both kind and GCP test environments/clusters. For kind environments, it does not yet provision one from scratch but depends on a developer/contributor to set one up themselves. That can be changed going forward. This commit adds:

  • a test framework to orchestrate dependencies centrally during a test run.
  • implementation for setting up GCP DNS dependencies for a test environment
  • additional terraform environment to bring up the test environment inside one of our GCP dev accounts
  • two (initial) test-specs: cluster (de)provisioning + namespace (de)provisioning.
  • changes to the opstracecluster module to allow for not creating GOB cluster(s) automagically

I refrained from adding more test-specs to this initial commit to limit its scope and allow for incremental development, though they have been developed and tested already. I'll add them in following commits, also start consolidating e2e tests from other parts of our repository. There remains some work on making this run in a CI pipeline, but more around it in a following commit as well.

Tested both with a kind and a GCP cluster, following is the output of a test run:

➜  opstrace git:(abhatnagar/e2e-framework) ✗ make e2e-test-suite
make -C test/e2e e2e-test
make[1]: Entering directory '/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e'
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/bin/ginkgo -v ./...
=== RUN   TestE2E
Running Suite: E2E Test Suite - /Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e
====================================================================================================
Random Seed: 1674830002

Will run 4 of 4 specs
------------------------------
[BeforeSuite]
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/e2e.go:40
  STEP: initialising test framework @ 01/27/23 15:33:22.612
  STEP: running tests for CI_COMMIT_SHORT_SHA=dealk @ 01/27/23 15:33:22.612
  STEP: running tests for TARGET=kind @ 01/27/23 15:33:22.612
  STEP: exporting KUBECONFIG= @ 01/27/23 15:33:22.612
  STEP: checking that the API server can be consistently queried @ 01/27/23 15:33:23.123
  STEP: checking that the scheduler-manager pod is ready & running @ 01/27/23 15:33:33.124
[BeforeSuite] PASSED [10.526 seconds]
------------------------------
cluster provisioning on creating an example cluster ensures it can be created successfully
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/cluster/cluster.go:47
  STEP: check Cluster CR has ready condition: True @ 01/27/23 15:33:33.179
  STEP: check all pods are ready in namespace: default @ 01/27/23 15:38:17.536
• [299.792 seconds]
------------------------------
cluster provisioning on deleting the example cluster ensures it can be deleted successfully
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/cluster/cluster.go:55
  STEP: gather all resources that are owned by this resource @ 01/27/23 15:38:32.93
  STEP: delete the CR: /dev-cluster @ 01/27/23 15:38:33.169
  STEP: wait for 114 owned resources to be deleted @ 01/27/23 15:38:33.194
• [29.208 seconds]
------------------------------
namespace provisioning on creating a GitlabNamespace object should reconcile a tenant if needed, a group and related resources
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/namespace/namespace.go:70
  STEP: check Cluster CR has ready condition: True @ 01/27/23 15:39:02.163
  STEP: check all pods are ready in namespace: default @ 01/27/23 15:43:51.716
  STEP: check GitLabNamespace CR has ready condition: True @ 01/27/23 15:44:22.174
  STEP: check all pods are ready in namespace: 9970 @ 01/27/23 15:45:55.375
• [424.497 seconds]
------------------------------
namespace provisioning on deleting a GitlabNamespace object should reconcile the tenant, delete the group and related resources
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/namespace/namespace.go:149
  STEP: gather all resources that are owned by this resource @ 01/27/23 15:46:46.82
  STEP: delete the CR: /dev-cluster @ 01/27/23 15:46:47.091
  STEP: wait for 114 owned resources to be deleted @ 01/27/23 15:46:47.109
• [69.204 seconds]
------------------------------
[AfterSuite]
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/e2e.go:119
[AfterSuite] PASSED [0.000 seconds]
------------------------------

Ran 4 of 4 Specs in 833.228 seconds
SUCCESS! -- 4 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestE2E (833.23s)
PASS

Ginkgo ran 1 suite in 14m5.091459375s
Test Suite Passed
make[1]: Leaving directory '/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e'
Edited by Ankit Bhatnagar

Merge request reports