feat(oci): container_virtual_repositories schema (S32 plan: 1/19)

What

Creates container_virtual_repositories: the parent table, its 64 hash partitions on namespace_id, the composite primary key (id, namespace_id), its two foreign keys with opposite delete actions, its unique index, and a CHECK holding id to UUID version 7.

  • repository_id references repositories (id, namespace_id) and cascades, so deleting the parent repository removes this row.
  • namespace_id references namespaces (id) with NO ACTION, left implicit, which every container table follows.
  • The unique index is unique_cvr_namespace_id_and_repository_id. The table abbreviates to cvr for consistency across the family, not because of the 63-character identifier limit: unique_container_virtual_repositories_ns_id_repository_id spells this table out at 57 characters and fits. What decides it is the two sibling tables ADR-007 puts next, which have no form that fits — container_virtual_repository_upstreams is 38 characters, so its ordered-position unique constraint is 107 spelled out with the _and_ separator this family uses, and still 75 with both columns abbreviated. Those names must abbreviate the table whatever this one does, so one scheme covers all three. The margin rule in docs/dev/database.md is a separate case, and the migration comment says so explicitly rather than relying on it.
  • The id CHECK is what ADR-007 requires of every uuid id with no server-side default. Handbook commit 22ced257 added that rule; the local ADR mirror carries it once the daily sync copies the commit across.

The table holds no other columns. Name, visibility, and the cross-format fields live on the parent repositories row.

Growth past the plan's Step 1 entry

Two things here are wider than the plan's Step 1 entry as merged, and both are deliberate.

  • The id version CHECK and its three tests. The entry's Scope named the parent, 64 partitions, the composite primary key, the two foreign keys, and the unique index — no CHECK. Handbook commit 22ced257 added the ADR-007 rule requiring it after this plan merged, so the omission is the plan's age, not a decision against it.
  • docs/specs/S32-container-virtual.md, which is in no step's Files entry. The hunk records the CHECK in the Data Model table, corrects which foreign-key edge ADR-007 leaves unstated, and retires both ADR-007 and ADR-009 amendment follow-ups: handbook !20928 and !20929 merged on 2026-09-01, after this branch's earlier spec edit, so the spec was sending a reader to open amendments that already landed. ./scripts/adr-freshness.sh reports the mirror stale until the daily sync copies both across, and the spec now says so instead of asserting the pre-amendment state.

!2160 (merged) books both in the plan, from its own docs(plans) branch, because guardrail 4 gives the plan file a single writer and a step branch may not edit it.

Why the diff is past 500 reviewable LOC

docs/dev/development-model.md asks for a split or a justification. A split does not help here: the migration and its schema suite are one unit, and the suite is what proves the migration.

Measured with git diff origin/main...HEAD --numstat at d07a7714e, summing added plus removed.

Group Lines Reviewable
container_virtual_repositories_schema_integration_test.go 1,589 yes
sql/20260831055729_create_container_virtual_repositories.sql 719 yes
structure.sql 1,235 no, regenerated
jet/artifact_registry/public/{model,table} and table_use_schema.go 103 no, regenerated
docs/dev/database-migrations.md 70 yes
docs/specs/S32-container-virtual.md 37 yes
internal/datastore/lifecycle_reap_repository.md 33 yes
migrations_checksum_test.go, repositories_integration_test.go 3 yes

2,451 reviewable lines, 1,338 generated, 3,789 total. Of the migration's 719 lines, 380 are comments: the file records why each absent column is absent, and scripts/ci/check-migration-immutability.sh freezes the text on merge, so adding the reasoning later takes an Allow-Migration-Edit override rather than an ordinary edit.

Merge-order note

!2106 (merged) (chore(datastore): bound every application-generated id to UUIDv7) touches two of the same paths: structure.sql and migrations_checksum_test.go. Whichever lands second conflicts on both, and no pipeline reports it. The conflicts are mechanical — a knownHeadVersion bump and regenerated DDL — so the second author regenerates and re-bumps. !2106 (merged)'s own merge-order section names the same two.

This MR has to land first. Its migration is 20260831055729 and !2106 (merged)'s is 20260831103000, so this one sorts earlier. internal/datastore/migrations/migrations.go:76 passes goose.WithAllowOutofOrder(false), which the function's doc comment at :50 states as "Sequential ordering only (out-of-order migrations are rejected)". An environment that applied !2106 (merged) first therefore refuses this migration.

!2106 (merged) records the same order, in its own ### Merge order against !2159 section: "!2159 (merged) should merge first", on the same goose reasoning. The constraint is therefore stated on both MRs, so either reviewer can check it without reading the other's diff.

No constraint-name collision arises in either order: !2106 (merged)'s migration names 38 tables explicitly and container_virtual_repositories is not among them. Its suite derives the table set from pg_class at run time and requires a validated check_<table>_id_uuid_version on each; this table's inline CHECK is validated on creation and its definition matches, so it satisfies that suite once both are on main.

If the order slips, a rebase alone does not fix it. The refusal is keyed on the version already in goose_db_version, not on the embedded set, so an environment that applied 20260831103000 refuses 20260831055729 however this branch is rebased. internal/gooseutil.UpVersions returns before it builds an apply list, so nothing in the pass is applied, Runner.Start returns an error, and Runner.Check stays on ErrMigrationsPending for the life of the pod — every pod, on the boot path. Recovery is to renumber this migration above 20260831103000 and re-bump knownHeadVersion (migrations_checksum_test.go:105), which is the tree's only reference to the timestamp.

Resolving the migrations_checksum_test.go conflict by bumping knownHeadVersion alone leaves TestHeadVersion green and the refusal live, so the test is not the guard here.

lock_timeout and #548

The Up sets no lock_timeout. #548 settled that schema-wide decision on 2026-08-10: the provider sets the value at 5-10s with a bounded retry over the existing advisory session lock. It was then closed on 2026-09-01 against that original ask, which left the Up-side implementation with no owner — the Down half of the decision lands in docs/dev/database-migrations.md in this MR and had not landed at closure. #548 is now reopened and re-scoped to the Up, so the 25 migrations that cite its URL resolve to a live issue again. This file's header states the decision beside that link rather than sending a reader to the tracker for the answer, because scripts/ci/check-migration-immutability.sh freezes the text on merge.

Twenty-five other migrations cite the same issue, and eight of those create a partitioned parent (of 43 that do, counting this one). Measured at d07a7714e, matching PARTITION BY on DDL lines only: a plain grep -lE 'PARTITION BY (HASH|RANGE|LIST)' also matches the phrase inside these files' comments and overcounts to eleven. The squawk-ignore-file require-timeout-settings directive in this file's header carries the reasoning.

The Down does set one, because its parent DROP takes ACCESS EXCLUSIVE on namespaces and the whole repositories tree, which blocks reads rather than only writes. 20260829134210_create_npm_virtual_upstream_rules already bounds its own Down the same way, so this is the second file in that shape, not a new pattern.

What the bound actually bounds, stated in the file rather than left to be assumed. PostgreSQL applies lock_timeout separately to each lock acquisition attempt, and the header's pg_locks sampling gives the parent DROP 66 relations to take ACCESS EXCLUSIVE on — namespaces plus the 65 of the repositories tree. So the statement's worst case is 66 × 5s, not 5s, and while it waits on the Nth it already holds the first N-1. No statement_timeout is added here: capping the statement is the schema-wide call #548 owns, and the per-acquisition bound still converts a stall that lasts as long as a conflicting session into one that ends. The Down comment carries the arithmetic so the next reader does not read 5s as the cap on the read stall.

The RESET now carries the caveat its cited sibling already spelled out: goose abandons a NO TRANSACTION migration at the first statement error, so a DROP that hits the bound never reaches the RESET, and the setting outlives the migration on whatever connection ran it.

Two consequences for whoever implements the Up half, both recorded on #548. The provider has to supply the value as a connection startup option rather than a SQL SET, because RESET restores the session-start value and would wipe a SQL-set one for the remainder of that pass (measured on PostgreSQL 16.14). And lock_timeout bounds each lock acquisition rather than the statement, so the value has to be chosen against the Up's worst case of one acquisition per FK target, not against a single wait.

docs/dev/database-migrations.md is corrected in the same MR. Its rollback guidance said a rollback has no session to issue a SET in, so a Down had to take its bound from PGOPTIONS. That is wrong on both entrypoints: migrations.Down and DownTo pin one *sql.Conn for the pass, and through the goose CLI the SET still carries because goose issues statements sequentially and database/sql hands each one its single idle connection. The section now describes both, cites the in-SQL example, and keeps PGOPTIONS for a Down carrying no SET of its own. Two shipped migrations still repeat the old claim; this MR does not touch them.

Testing

container_virtual_repositories_schema_integration_test.go covers the partition count and name sequence, column types and nullability, the absence of defaults and of soft_deleted_at, both foreign-key delete actions, the unique index shape, the CHECK shape and its rejection of other versions, per-partition index inheritance, and the constraint-name arithmetic. Row-shape subtests drive every value of visibility, format, and kind as a positive hit. Further subtests pin the four readings the table comment records as open on id: any timestamp, any variant bits, id equal to either other column, and one id reused across two namespaces. A static test asserts the Down's SET/RESET pair and its position around the DROPs.

Diffed against the sibling schema suites for dropped subtests, per guardrail 6. All 26 TestNPMVirtualRepositories* functions matching internal/datastore/migrations/npm_virtual_repositories_*_integration_test.go — 11 in the schema file and 15 in the rowshapes one — have a counterpart here, and nothing was dropped: NoCheckConstraints becomes OneCheckConstraint, which is the one intended divergence, and six functions are net-new (PartitionIndexNamesAreServerGenerated, IDVersionCheckShape, IDVersionCheckRejectsOtherVersions, IDVersionCheckReachesOnlyTheVersion, IDIsUnconstrainedAcrossTheThreeColumns, DownBoundsLockWaits). A reviewer cannot see from the diff that this comparison happened, which is why it is stated.

No conformance tests apply: this step adds no Maven, npm, or OCI protocol behavior.

No e2e scenario is added or affected. The step creates no request path; the plan's Step 18 owns the end-to-end harness and the catalog entries.

Database Review Evidence

Migrations

Note

Timings are from CI (db:migrate matrix, goose verbose) against an empty database, in apply / rollback order per PG version. Production-scale validation via Database Lab is not yet available. See Database review evidence for the matrix rationale and how to read the numbers.

Measured on pipeline 2809060819, over branch head d07a7714e. Commits after that one touch only comments in this migration — stripping comment and blank lines gives the same file hash at d07a7714e and at the current head — so the figures still describe the DDL that ships.

Migration PG 16 PG 17 PG 18
20260831055729_create_container_virtual_repositories.sql OK (565.41ms / 254.92ms) OK (348.62ms / 253ms) OK (622.75ms / 278.73ms)

Migration notes: none, against the three thresholds migration-mode.md ## 3.4 Flag migration anomalies sets.

  • Version-specific regression asks whether the slowest version is over 2x the second-slowest. PG 18 at 622.75ms over PG 16 at 565.41ms is 1.10x. (Slowest over fastest is 1.79x, against PG 17's 348.62ms; that is the wider statistic and it is not the threshold's.)
  • Slow migration and boot-budget risk asks for over 1s on an empty database. The slowest apply is 622.75ms.
  • Up/down asymmetry: the Up runs about 2.2x the Down on every version. That is the expected shape here rather than an anomaly — the Up creates the parent, 64 partitions and the unique index, and the Down only drops them.

Each job runs === Up ===, === Down ===, then === Re-Up ===; the table reports the Up and the Down, and the Re-Up (553.07ms / 488.49ms / 824.55ms) is excluded. lint:migration-ordering and lint:migration-immutability both pass on this pipeline. Query mode did not run: the only changed Go files are jet generated code and two *_test.go files, so none carries a dispatch signature.

Related to #291

Edited by Radamanthus Batnag

Merge request reports

Loading
Loading