Eliminate duplicated or unneeded jobs for backport MRs and stable branches
Problem
Between the pre- and post-merge pipelines, there are duplicated or unneeded tests / jobs being run. This leads to slower pipelines overall for backport MRs.
Current suite strategy:
Gather current test suites and execution for re... (gitlab-org/quality/quality-engineering/team-tasks#3623 - closed)
Proposal
Identify and eliminate any duplicated or unneeded jobs to improve the pipeline execution time for backport MRs.
Tasks
-
Remove jest vue3 check quarantined
andjest-with-fixtures vue3 check quarantined
from backport MRs (Runtime: ~17m) -
(If we didn't remove all jest
jobs) Disable Jest coverage on stable branches (like we did in Disable jest coverage in MRs (gitlab-org/gitlab!164463 - merged) for merge requests) (could potentially save 5.8 minutes average per job) -
Remove graphql-verify
job from stable branches (6m) - [-]
Identify unneeded jobs for CI changes. (context: for a simple CI change, and entire pipeline is run)Predictive test detection is ongoing on this track: gitlab-org/quality/analytics&12 (closed) -
Investigate and remove semgrep-appsec-custom-rules
from backport MRs (MR)
Results
What was done | Effect |
---|---|
Fixed semgrep-appsec-custom-rules flaky issue. |
Fixing this intermittent job eliminates both retry interruptions (which cost ~2-5 minutes plus engineer context-switching) and occasional 10-minute extended runs across 3 pipeline tiers. While the impact isn't as dramatic as full pipeline re-runs, preventing even 10-20 weekly failures still saves hours of compute time and eliminates disruptive engineer interruptions, improving both CI/CD efficiency and developer flow. |
Removed two unnecessary jobs, jest vue3 check quarantined and jest-with-fixtures vue3 check quarantined from backport MRs |
Removed two jobs totaling 17 minutes from backport MR pipelines, reducing Tier 1 and Tier 3 pipeline durations by ~16% and saving approximately 14 hours of compute time weekly. |
Disabled Jest coverage in stable branches. | Disabling Jest coverage in stable branches reduces 18.1, 18.0, and 17.11 stable branch pipeline times by 5-36% (saving 5.8-40 minutes per run) and eliminated flaky jobs that intermittently added ~59-minute delays, resulting in approximately 78 faster and more reliable pipelines per month. This change saves an estimated 90-624 hours monthly in CI/CD time while significantly improving developer productivity by reducing wait times and pipeline failures. |
Edited by Dan Davison