Spike: Deploy GDK in a VM for quick test results in MRs
Rationale
When we run E2E tests in CI currently, we build and install GitLab before running the tests. This can add an hour or more of a delay before test results are available. Some options for speeding up that process are discussed in https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1276
In contrast, we use GDK in development and it takes only a few minutes to update and start. If we could use GDK in CI then we could run E2E tests in MRs and get feedback to engineers much much sooner.
That was investigated in https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1011 but resource constraints make it unlikely to be worthwhile to proceed using GDK in a container.
Updated proposal
With incremental compilation and some nonessential services disabled, resource usage was reduced enough to run tests against GDK via Docker.
Tasks
-
Develop a script that will install GDK, checkout a branch, start GDK, and then run the QA tests. -
Outline the next steps to turn the prototype into a full implementation: https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1402
Original Proposal
Trial deploying GDK as a test environment in a VM in GCP and run a subset of QA tests relevant to the changes in the MR.
Evaluate the speed of deployment and reliability of the test results to see if it's suitable and capable of providing test results substantially faster than is possible currently.
Tasks
- Develop a script that will install GDK, checkout a branch, start GDK, and then run the QA tests.
- Add a CI job that will deploy a VM and then run the script.
- Automate setup and teardown of VMs used by the script.
Caveats
GDK is a development environment, so it's not a substitute for running E2E tests in a production-like environment. Therefore, even if we run tests on GDK in MRs, we should continue to run the tests against omnibus-gitlab in master and against a Review App in MRs.