## Goal The more accurate the feature_categories are on test metadata, the more likely we are to assign the any related tasks (e.g. flaky test quarantine MRs) to the correct person. For example, the ~"group::development analytics" team will implement auto-quarantining MRs for flaky tests to get assigned to an engineer who is likely to action. This epic is to ensure that tests with invalid feature_category metadata are fixed or removed so they can be processed by the Flaky test auto-quarantining process (and any other auto-assigning process). ## Improvements - Ensure `shared` category is disallowed for future tests. - Fix tests with invalid feature_categories - Fix/delete tests that are quarantined and have no feature_category metadata - Fix/delete tests that are quarantined and have :shared metadata - https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/4095+ - Note: ownerless/shared tests - will it be apart of a wider initiative to assign owners for app code also https://gitlab.com/gitlab-org/gitlab/-/issues/581839 (shared and not_owned) ## Current stats Snapshot of tests that have an invalid feature_category --- UNKNOWN CATEGORIES (17 unique) --- - ❌ :api 85 occurrences - ❌ :ci_scaling 5 occurrences - ❌ :compliance_managment 1 occurrence - ❌ :configuration 2 occurrences - ❌ :dependency_proxy 2 occurrences - ❌ :durability_metrics 20 occurrences - ❌ :feature_flag 1 occurrence - ❌ :group_management 1 occurrence - ❌ :integrations 317 occurrences - ❌ :internationalization 5 occurrences - ❌ :pipeline_execution 1 occurrence - ❌ :pipeline_reports 3 occurrences - ❌ :product_planning 1 occurrence - ❌ :scalability 45 occurrences - ❌ :snippets 1 occurrence - ❌ :tooling 146 occurrences - ❌ :webhooks 88 occurrences --- TOTALS --- - Total spec files scanned: 25024 - Unique categories found: 126 - ✅ Valid categories: 109 - ❌ Unknown categories: 17 - Specs with unknown categories: **724** (this excludes `shared` which is out of scope of this epic) --- ### Quarantined ownerless tests [Reference](https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/3959#note_2891664104) ``` [MISSING] 38 tests :shared 5 tests ``` The following MRs were compelted before the above stats were completed: - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/211836 - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/211515 - https://gitlab.com/gitlab-org/gitlab/-/merge_requests/211514 ## Exit Criteria - [x] **Prevent `shared` category**: Add validation/linting to disallow `:shared` feature_category in test metadata going forward - [x] **Fix invalid feature_categories**: Correct all tests that have invalid or non-existent feature_category values - [x] **Resolve quarantined tests without feature_category**: Fix or delete all quarantined tests missing feature_category metadata - [x] **Resolve quarantined tests with `:shared` metadata**: Fix or delete all quarantined tests that have `:shared` as their feature_category https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/4095 ### Participants -