We need all items in the release notes https://gitlab.com/gitlab-org/gitlab/-/tree/master/doc/releases to be listed in https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/features.yml ## Background and context For example in https://gitlab.com/gitlab-org/gitlab/-/blob/master/doc/releases/18/gitlab-18-11-released.md?ref_type=heads&plain=1#L22 these are the primary items: - Vulnerability resolution generally available on GitLab Duo Agent Platform - GitLab Data Analyst Foundational Agent now generally available - CI Expert Agent launches in beta - Automated vulnerability severity overrides - Create Service Account in subgroups and projects - Service Accounts available on GitLab Free - Fine-grained permissions for personal access tokens now available (Beta) - Top CWE chart in security dashboards - Deploy Gitaly on Kubernetes - Reconfigure inputs when manually running MR pipelines We expect PMs to merge individual notes, such as "Top CWE chart in security dashboards", in separate MRs. Often, they refer to existing features, so do not need a new entry. However, we need to block merging if the feature does not exist in `data/features.yml`. When release notes were published in www-gitlab-com, each release notes item was given a "category". That was validated against https://gitlab.com/gitlab-com/www-gitlab-com/-/blob/master/data/categories.yml which is the grouping list for `features.yml`. At minimum, we need to do the same `categories.yml` validation, though ideally we actually do `features.yml` validation. There are various possible options in how we trace the information: - https://gitlab.com/gitlab-org/gitlab/-/blob/master/config/feature_categories.yml has a list of all the `data/categories.yml` and all code is supposed to be tied to a category - work items typically have a `Category:` label, that label should match an existing category. However, the bot often auto-adds the category label based on the product group. It may be mislabelled if the work item is for a new category that hasn't been added yet. - something else I haven't thought of? --- ## Implementation options The following options were explored for validating that release note items map to valid feature categories. They can be combined for defense in depth. ### Option 1: HTML comment metadata + CI lint (Vale or custom script) PMs add the category as an HTML comment inside each item (invisible on the published page): ```markdown ### Top CWE chart in security dashboards <!-- category: vulnerability_management --> {{< details >}} ... {{< /details >}} ``` A CI check validates the comment value against `config/feature_categories.yml`. This could be implemented as: - **A Vale rule** scoped to `doc/releases/**/*.md` via a new `[doc/releases/**/*.md]` section in `.vale.ini` and a new `gitlab_releases` style. This runs in the existing `docs-lint markdown` CI job with no new pipeline infrastructure. - **A standalone lint script** (Ruby or shell) triggered by a CI job when `doc/releases/**` files change. - **Pros:** Deterministic, blocks merging on failure, self-contained within `gitlab-org/gitlab`, no API calls. Vale approach reuses existing CI infrastructure. - **Cons:** PMs must manually add the HTML comment (though the template can include it). ### Option 2: Validate via the linked work item's `Category:` label (API-based) A CI script parses the `Related issue`/`Related epic` URL already present in each item's `{{< details >}}` block, calls the GitLab API to fetch the work item's labels, extracts the `Category:` label, and validates it against `config/feature_categories.yml`. - **Pros:** No new metadata for PMs. Validates the actual work item, not a manually typed string. - **Cons:** Requires API token in CI. `Category:` labels can be auto-applied incorrectly. Adds external dependency on API availability. More complex to implement and debug. ### Option 3: Danger plugin A Danger plugin (similar to the existing `danger/feature_flag/Dangerfile`) runs on MRs touching `doc/releases/**` files. It can either check for an HTML comment category (like Option 1) or resolve the linked work item's labels (like Option 2), and posts a failure on the MR. - **Pros:** Fits the existing review workflow. Can combine both approaches. - **Cons:** Danger failures are sometimes ignored. If using API lookups, same complexity concerns as Option 2. ### Option 4: Duo Code Review custom rule Add an instruction group to `.gitlab/duo/mr-review-instructions.yaml` targeting `doc/releases/**/*.md`. Duo Code Review would check that each item's linked work item maps to a valid feature category and leave unresolved review comments if validation fails. Unresolved comments block immediate merging. - **Pros:** No new CI job. No extra metadata for PMs. Leverages existing Duo infrastructure. Soft enforcement that allows discussion on edge cases. - **Cons:** AI-based, not 100% deterministic. Depends on automatic Duo reviews being enabled. ### Option 5: Lightweight structural check (Vale or CI) + Duo custom rule Combine a deterministic check with an AI-based review: | Layer | Tool | What it checks | Enforcement | |---|---|---|---| | Static lint | Vale rule or CI script (scoped to `doc/releases/`) | Every item has a `Related issue`/`Related epic` link in its details block | Hard block (pipeline failure) | | AI review | Duo custom rule in `mr-review-instructions.yaml` | Category validity against `config/feature_categories.yml` | Soft block (unresolved comment) | The static check requires no new metadata from PMs and catches the most common oversight (missing work item links). The Duo rule handles the nuanced category validation on top. - **Pros:** Defense in depth. Static check is simple and deterministic. Duo handles judgment calls. No new metadata burden on PMs. - **Cons:** Two mechanisms to maintain. ### Option 6: Full `features.yml` validation (cross-repo) Validate that each item maps to an actual entry in `www-gitlab-com`'s `data/features.yml`, not just a valid category. Either periodically sync `features.yml` into `gitlab-org/gitlab`, or fetch it via API at CI time. - **Pros:** Strongest validation; ensures the feature actually exists in the product database. - **Cons:** Most complex. Cross-repo dependency. Feature titles don't match `features.yml` entries exactly, so fuzzy matching or explicit keys would be needed. --- ## Recommendation - **Option 4 (Duo Code Review custom rule)** is the easiest to ship: a single YAML change to the existing `.gitlab/duo/mr-review-instructions.yaml`, with no new CI jobs, scripts, or PM metadata. Unresolved Duo comments block immediate merging. However, it is non-deterministic (AI-based), so it may not catch every case and could be considered a first iteration rather than a long-term solution. - **Option 1 (HTML comment metadata + Vale rule)** is the most lightweight deterministic option. A Vale rule scoped to `doc/releases/**/*.md` validates the `<!-- category: ... -->` comment against `config/feature_categories.yml`, running in the existing `docs-lint markdown` CI job with no new pipeline infrastructure. It provides a hard pipeline gate. The tradeoff is that PMs must add the HTML comment, but the release notes template can be updated to include it. Note: this validates against categories, not the full `features.yml`. - **Option 6 (full `features.yml` validation)** is the only option that fully meets the stated goal of blocking merging if the feature does not exist in `data/features.yml`. All other options validate against categories only, which is the minimum bar. However, Option 6 is the most complex due to cross-repo dependencies and the need for fuzzy matching or explicit feature keys.