Validate: Remove Long Running test-on-omnibus Pipeline Setup for Backport MRs
Problem
Validate if e2e:test-on-omnibus-ee is executed on backport MRs, and transition them if so.
As identified in gitlab-org/quality/quality-engineering/team-tasks#3623 (closed) -
End-to-end suites
e2e:test-on-omnibus-eefull suite in tier 3
qa:test-on-gdkgdk-instance suite in tier 3
Context
Omnibus and CNG pipelines have significantly different runtimes
From our pipeline efficiency track:
- 
⏲️ ~95 minutes saved per MR pipeline on its critical path (~140 minsomnibus-> ~45 minscngDashboard)
- 
💵 ~10'000USD/month saved (calculation)
- 
🧱 Less pipeline false failures due to ability to block MRs based on test failures. Previously this gate was loosely added.
Proposal
To save time, money, and false failures, transition any e2e:test-on-omnibus type pipelines to e2e:test-on-cng if possible.
Metrics
Baseline e2e:test-on-omnibus-ee as of 2024-05-02:
Tier 1 (ref): ~95m Tier 3 (ref): ~95m
Tasks
- 
Remove e2e:test-on-omnibus-eefrom Tier 1 pipelines. (and backport to past 2 releases)
- 
Change e2e:test-on-omnibus-eeTier 3 to run smoke-only and orchestrated scenarios. (and backport to past 2 releases) - Change `e2e:test-on-omnibus-ee` Tier-3 to run s... (#21285 - closed)
- 
Replace e2e:test-on-gdkto usee2e:test-on-cngon Tier 3 pipelines to runinstancescenarios.(and backport to past 2 releases) - Replace e2e:test-on-gdk to use e2e:test-on-cng ... (#21286 - closed)
Results
| Task | Result | 
|---|---|
| Removed e2e:test-on-omnibus-eefrom tier-1 backport MR pipelines | Reduces tier 1 pipeline runtime ~40% without compromising quality. | 
| Removed instancetest suite (main e2e test suite) fromtest-on-omnibus | Reduces costs by not running same test suite against several deployment types, reduces flaky failure probability | 
| Removed test-on-gdkand replaced it withtest-on-cng | Higher confidence in patch release as it now uses actual customer like deployment instead of dev deployment, P80 feedback time improvement by 20 minutes which is the difference between test-on-cngandtest-on-gdkpipeline runtimes | 
Edited  by andrey