Explore Running CustomersDot E2E Tests in GitLab MRs
Problem
We currently have two sets of Fulfillment E2E test suites that live in different projects:
- GitLab: https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/qa/specs/features/ee/browser_ui/11_fulfillment
- CustomersDot: https://gitlab.com/gitlab-org/customers-gitlab-com/-/tree/main/qa
Both E2E test suites rely heavily on integration between GitLab, CustomersDot and Zuora.
There is currently no way to verify how incoming GitLab changes will affect CustomersDot E2E tests until the changes hit Staging, where it is often too late to catch any bugs before they make it to production. This also places most of the responsibility on Fulfillment SETs to update stale tests and triage test failures, rather than having developers catch these changes in their MRs earlier in the process.
Some tests can be modified to run locally (if you have a local CustomersDot setup), but this is very difficult and time consuming to do and most non-Fulfillment engineers would not have this setup in place.
Proposal
This issue should be used to explore how we can run CustomersDot E2E tests against a GitLab instance (containing the changes from a GitLab MR) integrated with a CustomersDot instance (relies on containerization work being completed in Run CustomersDot end-to-end tests per Customers... (gitlab-org/customers-gitlab-com#4384 - closed) first)