<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/engineering/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Label this issue](https://contributors.gitlab.com/manage-issue?action=label&projectId=278964&issueIid=624044) </details> <!--IssueSummary end--> ## Status - [x] Goal 1: the ruleset is a file in common-ci-tasks released with its tags. Merged in https://gitlab.com/gitlab-com/gl-infra/common-ci-tasks/-/merge_requests/1954 and released as v5.37.0. - [ ] Goal 2: a CI component runs the lint in MR, default-branch and tag pipelines. Child task 1. - [ ] Goal 3: a release cannot publish without a passing lint. Child task 2. - [ ] Goal 4: a service adds its own rules but cannot weaken the standard set. Child task 1 (part of the component). - [ ] Goal 5: Artifact Registry uses the component and passes. Child task 4. - [x] Goal 6: new Lab Bench features start with a rendered Redocly config. Merged in https://gitlab.com/gitlab-com/gl-infra/common-template-copier/-/merge_requests/1069 and released in common-template-copier v1.85.0. `bench scaffold` renders it once bench moves `BENCH_TEMPLATE_REF` from v1.83.0 to v1.85.0 or later. Child tasks: 1. Build the OpenAPI lint CI component in common-ci-tasks: https://gitlab.com/gitlab-org/gitlab/-/work_items/633601 2. Gate release-platform publishing on the OpenAPI lint: https://gitlab.com/gitlab-org/gitlab/-/work_items/633602 3. Add a Renovate rule that bumps the OpenAPI ruleset pin: https://gitlab.com/gitlab-org/gitlab/-/work_items/633603 4. Move Artifact Registry to the OpenAPI lint component: https://gitlab.com/gitlab-org/gitlab/-/work_items/633604 The technical considerations below are kept for history, and each child task carries the ones it needs. ## Problem ADR 008 (status: Proposed) says the API Platform team owns the OpenAPI quality bar and runs it through a CI component. Today the proposed ruleset is only a `redocly.yaml` uploaded to [an issue comment](https://gitlab.com/gitlab-org/gitlab/-/work_items/624043#note_3928667883). It is in no repository and has no version. No CI component runs it, so no service can adopt the bar. ADR 008: https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/theseus_platform_vision/decisions/008_openapi_specification_for_modular_http_apis/ ## Goal Give services one ruleset and one component to lint their OpenAPI specification, as ADR 008 points 3, 5 and 7 describe. 1. The ruleset is a file in `gitlab-com/gl-infra/common-ci-tasks`, released with that repository's tags. Each service pins it to a full release tag (for example `v5.37.0`), never a minor tag like `v5.37`, because minor tags move to the latest patch release (`v5.30` moved from `v5.30.0` to `v5.30.1`). 2. A CI component runs the lint in merge request, default-branch and tag (release) pipelines. 3. A release cannot publish without a passing lint (see decision D). 4. A service adds its own rules on top without copying the standard set, and cannot weaken it (see decision B). Artifact Registry's `x-gitlab-lifecycle` rule is the test case. 5. Done when [Artifact Registry](https://gitlab.com/gitlab-org/ops/artifact-registry) uses the component and passes. That means: - Include the component and delete its own `lint:openapi` job. - In `api/openapi/redocly.yaml`, replace only the top-level `extends: [minimal]` and `rules:` with an `extends` of the standard ruleset URL. Keep the `apis` entries and the per-API `rule/operation-lifecycle` rule, because the preview and Pages build use the `apis` entries. - Move its CI Redocly pin from 2.34.0 to 2.54.3 (the evaluation passed at 2.54.3). `GL_REDOCLY_CLI_VERSION` stays because the Pages `build-docs` job also uses it. 6. New Lab Bench features start with a Redocly configuration, rendered by common-template-copier, that extends the ruleset at a full release tag, so a team never writes its own. The file is skipped on `copier update`, so each service owns and moves its own pin. Template MR: https://gitlab.com/gitlab-com/gl-infra/common-template-copier/-/merge_requests/1069 ## Decisions - A. How services get the ruleset: each service's own lint config extends the ruleset by a URL pinned to a full common-ci-tasks release tag, separately from the component version. Local runs then match CI. Cost: a second pin per service and a custom Renovate rule to bump the URL. Recorded in ADR 008 in https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/21444 - B. The floor is enforced. Redocly's later `extends` entries and `rules` overwrite earlier ones, so a service config can turn a standard error rule off. The component takes a `spec_paths` input and also lints those paths against the standard ruleset alone, so a weakened local config cannot pass. - C. The API Platform team owns the component. It gets an optional CODEOWNERS section in `gitlab-com/gl-infra/common-ci-tasks` covering the component's files, the same way the existing `^[Nix]` section covers the Nix component's files. The `^` marks a section as optional, so its owners are listed for those files but their approval is not required, and the existing `[Owners]` section still covers every file. - D. Release gating goes in release-platform. Today its publish job `generate-publish-pipeline` only needs `goreleaser`, and `goreleaser` has `needs: []`, so neither waits for validate-stage jobs and a failing lint does not stop a publish. This was read from the configuration, not seen in a pipeline. The fix is an optional `needs` on the lint job in release-platform's publish chain, at least on `generate-publish-pipeline`. Decide whether `goreleaser` also waits. ## Technical considerations - Component files follow common-ci-tasks' Jsonnet workflow: `src/ci/components/<name>.jsonnet` and `<name>.manitest.jsonnet`, a downstream test in `test/downstream-pipeline-tests/tests.<name>.yml`, a `<name>.md` doc, generated with `scripts/generate.sh`. The Redocly CLI version goes in `.gitlab-ci-other-versions.yml` as `GL_COMMON_CI_TASKS_REDOCLY_CLI_VERSION` with a Renovate comment. - Tag pipelines need new rules. `rules.standard()` blocks tags by default, and `allowTags=true` only removes that block; nothing makes the job run on a tag. Add a tag rule in the component or in the shared rules library. - Check the security mirror before relying on decision D. Releases run on the security mirror. If it counts as a mirror (`GITLAB_INFRA_MIRROR_PROJECT_URL` set), standard rules skip the lint job there by default. An optional `needs` is then satisfied by the missing job, so the publish runs with no lint at all. Either set `COMMON_CI_TASKS_VALIDATE_ON_MIRROR` or give the lint job its own rule for release pipelines. - Renovate and the pin: the shared `renovate-truncated-versions` preset has rules for the package `gitlab-com/gl-infra/common-ci-tasks` that truncate versions to major.minor in every manager. A bump rule for the ruleset URL that used this package name would rewrite the pin to a minor tag such as `v5.37`, which moves to each new patch release, and that breaks the decision that each service pins a full tag. The URL bump rule therefore needs a package name those truncation rules do not match, and the template's pin needs the same rule. The template's existing Renovate regex for `.jinja` files would also capture the whole URL as the version, so it cannot be reused as it is. - Bench pin: `bench` defaults to template `v1.83.0` through the `BENCH_TEMPLATE_REF` environment variable. `bench scaffold` renders the configuration file only after bench moves that pin to a template release that includes it. - Spike, Redocly CLI 2.54.3 against Artifact Registry's two documents: - Extending the ruleset by URL works, including the ruleset's own `extends: minimal`. - Results match the evaluation: 0 errors and 9 `operation-tag-defined` warnings on `gitlab-v1.yaml`. - The per-API lifecycle rule still applies on top. - A config that turns `operation-description` and `operation-operationId` off passes a spec missing both. Linting the same spec against the standard ruleset alone fails it. - The spike used the ruleset's issue-upload URL in place of a common-ci-tasks tag URL. Raw URLs at common-ci-tasks tags return 200. - Out of scope: - Breaking-change detection: https://gitlab.com/groups/gitlab-org/quality/-/work_items/432 - Route completeness (`bench build`, Lab Bench) - Adding the component's CI include to the scaffolded pipeline stays out of scope and is tracked in https://gitlab.com/gitlab-com/gl-infra/common-template-copier/-/work_items/20. The configuration file is item 6. - Users of this work: - https://gitlab.com/gitlab-com/gl-infra/common-template-copier/-/work_items/20 - https://gitlab.com/gitlab-org/gitlab/-/work_items/624045 - https://gitlab.com/groups/gitlab-org/quality/-/work_items/429 *Repurposed from "Cost out maintaining both definition paths", which ADR 008 superseded: it mandates no production method, so GitLab is not building generators per runtime.*