feat(testing): Introduce e2e testing framework for Gitlab Observability Platform
Related to: #1830 (closed), #1723 (closed), #1722 (closed)
This commit introduces a new framework to allow for running E2E tests against both kind
and GCP
test environments/clusters. For kind
environments, it does not yet provision one from scratch but depends on a developer/contributor to set one up themselves. That can be changed going forward. This commit adds:
- a test framework to orchestrate dependencies centrally during a test run.
- implementation for setting up GCP DNS dependencies for a test environment
- additional terraform environment to bring up the test environment inside one of our GCP dev accounts
- two (initial) test-specs:
cluster
(de)provisioning +namespace
(de)provisioning. - changes to the
opstracecluster
module to allow for not creating GOB cluster(s) automagically
I refrained from adding more test-specs to this initial commit to limit its scope and allow for incremental development, though they have been developed and tested already. I'll add them in following commits, also start consolidating e2e tests from other parts of our repository. There remains some work on making this run in a CI pipeline, but more around it in a following commit as well.
Tested both with a kind
and a GCP
cluster, following is the output of a test run:
➜ opstrace git:(abhatnagar/e2e-framework) ✗ make e2e-test-suite
make -C test/e2e e2e-test
make[1]: Entering directory '/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e'
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/bin/ginkgo -v ./...
=== RUN TestE2E
Running Suite: E2E Test Suite - /Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e
====================================================================================================
Random Seed: 1674830002
Will run 4 of 4 specs
------------------------------
[BeforeSuite]
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/e2e.go:40
STEP: initialising test framework @ 01/27/23 15:33:22.612
STEP: running tests for CI_COMMIT_SHORT_SHA=dealk @ 01/27/23 15:33:22.612
STEP: running tests for TARGET=kind @ 01/27/23 15:33:22.612
STEP: exporting KUBECONFIG= @ 01/27/23 15:33:22.612
STEP: checking that the API server can be consistently queried @ 01/27/23 15:33:23.123
STEP: checking that the scheduler-manager pod is ready & running @ 01/27/23 15:33:33.124
[BeforeSuite] PASSED [10.526 seconds]
------------------------------
cluster provisioning on creating an example cluster ensures it can be created successfully
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/cluster/cluster.go:47
STEP: check Cluster CR has ready condition: True @ 01/27/23 15:33:33.179
STEP: check all pods are ready in namespace: default @ 01/27/23 15:38:17.536
• [299.792 seconds]
------------------------------
cluster provisioning on deleting the example cluster ensures it can be deleted successfully
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/cluster/cluster.go:55
STEP: gather all resources that are owned by this resource @ 01/27/23 15:38:32.93
STEP: delete the CR: /dev-cluster @ 01/27/23 15:38:33.169
STEP: wait for 114 owned resources to be deleted @ 01/27/23 15:38:33.194
• [29.208 seconds]
------------------------------
namespace provisioning on creating a GitlabNamespace object should reconcile a tenant if needed, a group and related resources
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/namespace/namespace.go:70
STEP: check Cluster CR has ready condition: True @ 01/27/23 15:39:02.163
STEP: check all pods are ready in namespace: default @ 01/27/23 15:43:51.716
STEP: check GitLabNamespace CR has ready condition: True @ 01/27/23 15:44:22.174
STEP: check all pods are ready in namespace: 9970 @ 01/27/23 15:45:55.375
• [424.497 seconds]
------------------------------
namespace provisioning on deleting a GitlabNamespace object should reconcile the tenant, delete the group and related resources
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/namespace/namespace.go:149
STEP: gather all resources that are owned by this resource @ 01/27/23 15:46:46.82
STEP: delete the CR: /dev-cluster @ 01/27/23 15:46:47.091
STEP: wait for 114 owned resources to be deleted @ 01/27/23 15:46:47.109
• [69.204 seconds]
------------------------------
[AfterSuite]
/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e/e2e.go:119
[AfterSuite] PASSED [0.000 seconds]
------------------------------
Ran 4 of 4 Specs in 833.228 seconds
SUCCESS! -- 4 Passed | 0 Failed | 0 Pending | 0 Skipped
--- PASS: TestE2E (833.23s)
PASS
Ginkgo ran 1 suite in 14m5.091459375s
Test Suite Passed
make[1]: Leaving directory '/Users/ankitbhatnagar/go/src/gitlab.com/gitlab-org/opstrace/test/e2e'