> **Supersedes [&21862](https://gitlab.com/groups/gitlab-org/-/epics/21862)** — the original charter. See it for the project's motivation, the npm/Maven/OCI prior-art survey, and the original protocol-mode / auth-mode design. Delivers the **registry-conformance** tool for closed beta — a standalone suite that validates any Artifact Registry implementation against the Maven, npm, and OCI protocols from *outside* the service boundary. It ships in two consumption modes: a **CLI binary** for the CI gate, and an importable **`pkg/conformance` library** for AR's own integration tests. Closed-beta scope = the **S04 foundation harness** + the **S05 (Maven)**, **S06 (npm)**, and **S07 (OCI)** format slices. Each slice implements one `TestDescriptor` per non-`optional` Phase 1 catalog row, exercised over real native clients (`mvn`, `npm`, `crane`) and direct HTTP. The tool lives outside the registry codebase deliberately: agents implementing registry features cannot weaken a conformance test to mask a bug. The suite is authored against each protocol's spec and reference behavior (npmjs.org, Maven Central, the OCI reference), **not** against GitLab — so known GitLab gaps surface as failures, never as silently weakened rows. - **Specs:** [`S04-contracts`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/specs/S04-contracts.md) · [`S05-maven-protocol`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/specs/S05-maven-protocol.md) · [`S06-npm-contracts`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/specs/S06-npm-contracts.md) · [`S07-oci-protocol`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/specs/S07-oci-protocol.md) - **Plans:** [S04 foundation](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/plans/2026-05-08-s04-foundation.md) · [S05 Maven](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/plans/2026-05-20-maven-format.md) · [S06 npm](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/plans/2026-06-02-npm-format.md) · [S07 OCI](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/plans/2026-05-26-oci-format.md) — all merged. - **Catalog (test source of truth):** [`docs/catalog`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/tree/main/docs/catalog). - **Validates (closed-beta format workstreams):** Maven (S10, &22310) · npm (S11, &22301) · Container/OCI (S12, &22315). - **Gate policy:** hard gate — each format workstream must pass its **critical + high** conformance rows before closed-beta sign-off. - **Out of scope:** `optional` and `Virtual` catalog rows (await a follow-up spec); the management/scaffolding API (gitlab-org/gitlab#597641, separate track); a standalone auth mode / mock JWT issuer (in the &21862 charter but not in the current S04–S07 specs — revisit if closed beta needs it); per-format CLI flags (Phase 1 has none). ### Tracking Implementation is tracked as project work items in [`gitlab-org/ops/registry-conformance`](https://gitlab.com/gitlab-org/ops/registry-conformance), one umbrella issue per slice; each carries its own per-step status table and dependency DAG. The foundation gates all three format slices; the slices gate distribution and the library handoff. - :gear: [#28 Implement S04 Foundation Harness](https://gitlab.com/gitlab-org/ops/registry-conformance/-/work_items/28) — in progress (16/25 steps done, 4 in progress). - :package: [#29 Implement S05 Maven Format Tests](https://gitlab.com/gitlab-org/ops/registry-conformance/-/work_items/29) — ready for development (27 steps). - :package: [#30 Implement S06 npm Format Tests](https://gitlab.com/gitlab-org/ops/registry-conformance/-/work_items/30) — plan merged (32 steps). - :package: [#31 Implement S07 OCI Format Tests](https://gitlab.com/gitlab-org/ops/registry-conformance/-/work_items/31) — plan merged (35 steps). - :rocket: [#32 Distribution: versioned binary + Docker image](https://gitlab.com/gitlab-org/ops/registry-conformance/-/work_items/32) — goreleaser binary + Docker image so CI consumers pull a pinned tool. - :books: [#33 Library-integration handoff](https://gitlab.com/gitlab-org/ops/registry-conformance/-/work_items/33) — stable `pkg/conformance` surface + a worked recipe for AR integration tests. ### Related work items - :clipboard: Implementation plans (Maven, npm, OCI) — merged (gitlab-org/gitlab#601208) - :scroll: Protocol specs *(closed)*: gitlab-org/gitlab#598527 · Project scaffold *(closed)*: gitlab-org/gitlab#597641 ### Cross-workstream dependencies (critical path) Conformance is **upstream** of each format workstream's conformance step — the runner must exist before that workstream can prove (or measure) conformance. It is authored against the protocol spec, not against AR, so it is **not blocked by** the registry workstreams. - **Maven (S10, &22310)** — the Maven runner (#29) gates &22310 Step 11; that epic already names this work as its conformance gate. Maven's RFC 9457 error envelope is in S10 scope, so `maven.error.rfc-9457-problem-details` should pass by sign-off. - **npm (S11, &22301)** — the npm runner gates &22301's S23 conformance step. Full + abbreviated packument is in S11 scope (closes `npm.packument.full`). **Open coordination with the npm DRI:** SRI `dist.integrity` (sha512) and the `_rev` field are not in S11's stated scope — decide whether to implement them or downgrade those catalog rows below the critical+high gate line. - **Container/OCI (S12, &22315)** — the OCI runner gates the OCI conformance step. No known GitLab gaps in the OCI catalog. - **Environment:** running the suite against AR in CI needs a deployed AR instance with the format enabled — a runtime dependency on each format workstream, not a build-time blocker for this tool. ## High-Level Implementation Plan <details> <summary>Expand — carried over from &21862, updated to current state</summary> **Layout & layering:** see [`docs/dev/architecture.md`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/blob/main/docs/dev/architecture.md). (Supersedes the original directory sketch; the test source of truth moved from `docs/roadmap.md` to [`docs/catalog`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/tree/main/docs/catalog).) ### Decisions | Decision | Choice | |----------|--------| | Project name | `registry-conformance` | | Public `pkg/` | Yes — importable client library; AR consumes it programmatically in its integration tests | | OCI strategy | Native implementation; upstream OCI conformance suite as reference only | | Language | Go | | CLI framework | `urfave/cli/v3` | | Test framework | `testing` + `testify/require` | | Logging | `log/slog` (JSON handler) | | HTTP | Standard-library client | | Report format | JUnit XML + human-readable stdout | | Test isolation | Run-ID namespacing | | Scaffolding | Copier template | | Auth | One credential per run (valid creds); negative-auth tests run outside the credential loop | ### Dev model (AI-assisted) 1. Pick the next `not started` row from the relevant [`docs/catalog/<format>.md`](https://gitlab.com/gitlab-org/ops/registry-conformance/-/tree/main/docs/catalog). 2. Read the relevant protocol spec in `docs/specs/`. 3. Implement (`pkg/client/<format>` + conformance tests + wiring). 4. Validate against a live registry. 5. Flip the catalog row's status; the inventory test enforces it. For MRs that introduce or change test cases, the agent writes a plan in `docs/plans/` and the operator approves before implementation. </details>