Explore Running GitLab Fulfillment E2E Tests in CustomersDot MRs
Problem
We currently have two sets of Fulfillment E2E test suites that live in different projects:
- GitLab: https://gitlab.com/gitlab-org/gitlab/-/tree/master/qa/qa/specs/features/ee/browser_ui/11_fulfillment
- CustomersDot: https://gitlab.com/gitlab-org/customers-gitlab-com/-/tree/main/qa
Both E2E test suites rely heavily on integration between GitLab, CustomersDot and Zuora.
There is currently no way to verify how incoming CustomersDot changes will affect GitLab E2E tests until the changes hit Staging, where it is often too late to catch any bugs before they make it to production. This also places most of the responsibility on Fulfillment SETs to update stale tests and triage test failures, rather than having developers catch these changes in their MRs earlier in the process.
Some tests can be modified to run locally (if you have a local CustomersDot setup), but this is very difficult and time consuming to do and most non-Fulfillment engineers would not have this setup in place.
Proposal
This issue should be used to explore how we can run GitLab Fulfillment E2E tests against a GitLab instance integrated with the changes contained in a CustomersDot MR (relies on containerization work being completed in Run CustomersDot end-to-end tests per Customers... (gitlab-org/customers-gitlab-com#4384 - closed) first)