Discussion in https://gitlab.com/gitlab-org/quality/analytics/team/-/issues/17 ## Goal Stop running E2E tests against the GDK (`e2e: test-on-gdk`) and rely on `test-on-cng` (and `test-on-omnibus`) instead, then retire the GDK E2E pipeline. This would bring: - cost savings - less friction - more reliable pipelines ## Rationale - We are essentially running all E2E tests twice (GDK + CNG), which adds time and cost to monolith pipelines - Test failures have shown to be consistent across CNG and GDK - Building the GDK image is taking too long - More chance of a flaky test when we run the E2E test pipeline multiple times - CNG can be run locally, is more production-like than the GDK's monolithic dev topology, and GDK will be replaced by Caproni - Engineers can still use GDK (we accept the risk that there may be CNG-specific failures that cannot be reproduced against the GDK, although this hasn't occurred much at all) ## Blocking requirements ### 1. E2E suites that run only in `test-on-gdk` (port or retire) Each GDK-only suite must be either ported to CNG or retired before GDK can be turned off: - [x] **Inverse feature-flag job** — ported to CNG (`cng-instance-ff-inverse`). → #4557 (closed) - [x] **`gitaly-transactions` suite** — **Retired.** ~"group::gitaly" confirmed this coverage was no longer required, so the suite was removed rather than ported to CNG. → #4558 · removed in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/248226 (merged) - [x] **`gdk-instance-with-load-balancer` (Postgres load balancing) job** — **Retired.** The job never ran automatically anywhere (`when: manual` + `allow_failure`, excluded from schedules, only added on load-balancing/GDK-CI diffs) and there is no longer a ~"group::test governance" team to maintain it, so it was removed rather than ported. LB logic remains covered by ~28 unit/integration specs; follow-up on whether to add system/E2E coverage: gitlab-org/gitlab#607951. → #4594 · removed in https://gitlab.com/gitlab-org/gitlab/-/merge_requests/248228 (merged) ### 2. Other requirements - [x] **`http-router` coverage** — **Resolved / not a blocker.** ~"group::cells" agree the focus should be robust contracts between the monolith and the HTTP router rather than a full-monolith GDK E2E run (the router issues we've seen originated from router changes, not monolith changes). http-router support on Caproni is in progress: https://gitlab.com/groups/gitlab-com/gl-infra/tenant-scale/cells-infrastructure/-/work_items/35. Original context: https://gitlab.com/gitlab-org/quality/analytics/team/-/issues/17#note_2479818599 {width=900 height=388} ## Dependencies: E2E coverage data produced only by test-on-gdk Collection for both feeds below is scheduled-only (nightly `if-dot-com-gitlab-org-schedule`) — neither runs in MRs. See also https://gitlab.com/groups/gitlab-org/quality/test-governance/-/epics/5#note_2972194571 **1. Code-paths mapping (affected-spec selection) — being ported.** `export-code-paths-mapping` (backend, Coverband) + `export-frontend-code-paths-mapping` (frontend, Istanbul) produce the E2E→code-paths mapping that `qa_changes.rb` consumes (keyed on `e2e-test-on-gdk`) to select the **affected-only** E2E specs for an MR diff. It **is** consumed, so it must be preserved on CNG before turn-off. - [ ] Port `export-code-paths-mapping` / `export-frontend-code-paths-mapping` to `e2e:test-on-cng` and repoint `qa_changes.rb` to the CNG mapping → https://gitlab.com/gitlab-org/gitlab/-/work_items/609350 **2. `process-e2e-coverage` (per-test coverage → ClickHouse) — TBD.** Merges Coverband + Istanbul E2E coverage into per-test coverage consumed by the main pipeline's `test-coverage:export-rspec-and-e2e` / `test-coverage:export-jest-and-e2e` jobs (→ ClickHouse). Whether this is still needed is being confirmed (awaiting reply). - [ ] Confirm whether `process-e2e-coverage` per-test coverage is consumed; port to CNG or drop accordingly. ## Final step Blocking requirements (section 1 + `http-router`) are resolved. The remaining dependency before turn-off is the code-paths mapping port above (https://gitlab.com/gitlab-org/gitlab/-/work_items/609350). - [ ] Remove the `e2e: test-on-gdk` pipeline and supporting GDK image build. (Turn-off started: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/249283) - [ ] Update the E2E pipeline monitoring handbook page to remove/adjust `test-on-gdk` references: https://gitlab.com/gitlab-com/content-sites/handbook/blob/main/content/handbook/engineering/testing/end-to-end-pipeline-monitoring.md - [ ] Mark the E2E pipeline diagram on that handbook page as outdated (it still shows `test-on-gdk`). - [ ] Update the [`disable-the-e2etest-on-gdk-pipeline` runbook](https://gitlab.com/gitlab-org/quality/runbooks/-/tree/master/test_on_gdk#disable-the-e2etest-on-gdk-pipeline) once `test-on-gdk` is removed.