[_Engineering initiative_](https://handbook.gitlab.com/handbook/engineering/#engineering-initiatives) **Owning team:** Vulnerability Management ## TL;DR Get the quarantined `vulnerability_management` tests back to delivering real CI value. Fix them in place where we can. When a test is genuinely unfixable or shouldn't live at the E2E layer anymore, move that coverage down to MSW or another [appropriate testing level](https://docs.gitlab.com/development/testing_guide/testing_levels/). ## Why The vulnerability management domain has a large share of quarantined and flaky E2E tests. 57% (12/21) are quarantined (see the [Audit](#audit) for the full breakdown), which means: 1. **Real coverage gaps**: quarantined examples are skipped by the test runner, so the QA pipeline reports success without ever exercising these areas. Regressions surface in production instead of CI. 2. **Dead weight in the codebase**: ~57% quarantined spec blocks in `vulnerability_management` that aren't being run. 3. **Maintenance**: hurdles for implementing new features or refactoring. Many tests also still contain conditional logic for finished migrations. This initiative is scoped to the Vulnerability Management domain. Within it, we will: 1. **Fix in place**: the default first attempt for any quarantined, outdated, long-running, or hard-to-maintain E2E spec. 2. **Replace with lower-layer coverage (MSW or other)**: when fix-in-place isn't viable, when the E2E test doesn't belong at that layer, or when the cost of keeping it at E2E outweighs the value. 3. **Showcase MSW patterns**: since we currently have zero MSW coverage in this domain, even one well-built MSW spec from path 2 doubles as a reusable template. 4. **Capture learnings and explore AI agent capabilities**: skills, memory, hooks, and subagents that take repeatable parts of this work off our hands. ### MSW [MSW integration tests](https://docs.gitlab.com/development/testing_guide/frontend_testing/#msw-integration-tests) give us an extra layer between Jest unit tests and Capybara feature specs: full Vue app mount in jsdom with intercepted GraphQL/REST. It should run way faster, with no flake. For example, a suite that used to take ~5 minutes can run in ~13 seconds. ## Goals 1. **Reduce quarantined QA specs in `vulnerability_management` to below 10%**: fix, replace with lower-layer coverage, or delete. 2. **Put each test at the right layer**: for every quarantined test we touch, justify why E2E is the right home, or push the coverage down to a faster, less flaky layer (MSW). 3. **Share learnings**: as written notes for other domains to apply, and where it pays off, as AI skills, memory entries, or hooks the team can keep using. ## Non-goals - Replacing E2E coverage that genuinely requires browser APIs (web workers, websockets) or cross-app journeys. - Touching domains outside `vulnerability_management`. If other teams want to apply the same approach in their domain after seeing how this goes, that's a separate initiative. - Mass-skipping. `skip:` instead of `quarantine:` is not an outcome: every test gets fixed, replaced, or deleted intentionally. ## Definition of done (per test) Each quarantined test exits via one of three paths. All require tracking issue closed. - **Fix in place**: root cause identified; fix passes consistently in CI (≥3 consecutive runs); `quarantine:` metadata removed; MR merged; test case closed. - **Replace**: coverage parity confirmed (MSW / Capybara / request / parser / ingestion); replacement merged; `quarantine:` metadata deleted; test case closed. - **Delete**: justification documented; `quarantine:` metadata deleted; test case closed. More info about how to [Dequarantine a test](https://handbook.gitlab.com/handbook/engineering/testing/quarantine-process/#dequarantine-a-test). ## Audit (June 11th 2026) ### Quarantined Vulnerability Management E2E tests Within `qa/qa/specs/features/ee/browser_ui/18_security_risk_management/`, filtered to `feature_category: :vulnerability_management`: | File | Examples | Quarantined | Notes | |---|---|---|---| | `security_reports_spec.rb` | 6 | **5** | Issues [1682](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/1682)/[1008](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/1008)/[1683](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/1683)/[1684](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/1684)/[24047](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/24047). Mix of `:flaky` and `:stale`. | | `vulnerability_management_spec.rb` | 2 | **2** | Both on issue [22185](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/22185) (`:flaky`). MR security widget + pipeline security tab flows. | | `vulnerability_security_training_spec.rb` | 2 | **2** | Both `:test_environment` (issue [24048](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/24048)): training endpoint env-dependent. | | `create_merge_request_with_secure_spec.rb` | 1 | **1** | `:flaky` issue [932](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/932) | | `dismissed_vulnerabilities_in_security_widget_spec.rb` | 1 | **1** | `:flaky` (issue [23729](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/issues/23729)). | | `fix_vulnerability_workflow_spec.rb` | 1 | **1** | `:flaky` (work item [39932](https://gitlab.com/gitlab-org/quality/test-failure-issues/-/work_items/39932)). | | `change_vulnerability_status_spec.rb` | 3 | 0 | Passing. | | `explain_this_vulnerability_spec.rb` | 1 | 0 | Passing. | | `export_vulnerability_report_spec.rb` | 1 | 0 | Passing. | | `project_vulnerability_report_spec.rb` | 2 | 0 | Passing. | | `vulnerabilities_jira_integration_spec.rb` | 1 | 0 | Passing. | **Totals: 21 examples, 12 quarantined (~57%).** ### Feature specs No quarantined feature specs. ### Quarantined E2E tests of all domains | Domain (feature_category) | Examples | Quarantined | % | |---|--:|--:|--:| | workspaces | 2 | 2 | 100% | | pages | 1 | 1 | 100% | | dynamic_application_security_testing | 1 | 1 | 100% | | duo_chat | 5 | 4 | 80% | | security_policy_management | 12 | 8 | 67% | | service_ping | 3 | 2 | 67% | | software_composition_analysis | 3 | 2 | 67% | | wiki | 19 | 12 | 63% | | **vulnerability_management** | **21** | **12** | **57%** | | global_search | 14 | 8 | 57% | | package_registry | 34 | 19 | 56% | | consumables_cost_management | 4 | 2 | 50% | | code_review_workflow | 27 | 11 | 41% | | continuous_integration | 32 | 13 | 41% | | code_suggestions | 22 | 8 | 36% | | plan_provisioning | 3 | 1 | 33% | | incident_management | 16 | 5 | 31% | | pipeline_composition | 35 | 10 | 29% | | environment_management | 9 | 2 | 22% | | organization | 33 | 6 | 18% | | compliance_management | 24 | 4 | 17% | | subscription_management | 8 | 1 | 13% | | source_code_management | 92 | 11 | 12% | | container_registry | 9 | 1 | 11% | | gitaly | 10 | 1 | 10% | | system_access | 36 | 3 | 8% | | team_planning | 45 | 3 | 7% | | portfolio_management | 15 | 1 | 7% | | importers | 35 | 0 | 0% | | geo_replication | 25 | 0 | 0% | | secrets_management | 23 | 0 | 0% | | value_stream_management | 11 | 0 | 0% | | cell | 8 | 0 | 0% | | runner_core | 7 | 0 | 0% | | web_ide | 6 | 0 | 0% | | observability | 3 | 0 | 0% | | secret_detection | 2 | 0 | 0% | | code_testing | 1 | 0 | 0% | | acquisition | 1 | 0 | 0% | ## References - Agent Skill [`qa-quarantine-triage`](https://gitlab.com/gitlab-org/security-risk-management/security-insights/si-agent-kit/-/blob/main/skills/qa-quarantine-triage/SKILL.md?ref_type=heads) to triage a single quarantined QA spec - [Dequarantine a test](https://handbook.gitlab.com/handbook/engineering/testing/quarantine-process/#dequarantine-a-test) - [Testing levels](https://docs.gitlab.com/development/testing_guide/testing_levels/) - [MSW](https://docs.gitlab.com/development/testing_guide/frontend_testing/#msw-integration-tests)