feat(maven): maven remote files schema (S14 Step 4)

What

Adds the maven_remote_files table — the file-level cache table for kind=2 (remote) Maven repositories, child of Step 2's maven_remote_packages and Step 3's maven_remote_versions — as Step 4 of the S14 Maven remote vertical slice. One goose migration, the regenerated jet types and structure.sql dump, an integration test suite asserting the schema shape and every constraint's accept and reject paths, an S14 spec amendment recording the divergences that harden maven_remote_files beyond ADR-007 (the file_name path-separator and dot-dot rejection, the upstream_etag control-character rejection, and the version-package composite FK), and a plan amendment correcting Step 4's Files entry to what actually landed.

No Go production code and no behavior change: nothing reads or writes this table until Step 8 (remote cache store Lookup) and Step 9 (cache-fill writes).

Schema

Follows the npm_remote_files boilerplate. As the slice's only file-bearing table it takes the cache-content Down shape (DROP per partition, no DETACH) rather than Steps 1–3's DETACH-then-DROP:

  • PARTITION BY HASH (namespace_id) × 64, children in the partitions schema.
  • PK (id, namespace_id); app-generated UUIDv7, no sequence or default.
  • Digest columns: blob_sha256, sha1, sha512 NOT NULL bytea with octet-length CHECKs (32/20/64); md5 nullable with a 16-octet CHECK, so a FIPS-mode fill writes NULL.
  • Nullable maven_remote_version_id splits version-level from package-level rows; composite FKs to maven_remote_packages(id, namespace_id) and — pinning the file's package to its version's package — maven_remote_versions(id, maven_remote_package_id, namespace_id), NO ACTION per ADR-007's artifact-table rule. MATCH SIMPLE skips the version composite FK when maven_remote_version_id is NULL (the package-level maven-metadata.xml case).
  • Both composite blob FKs active: (blob_storage_attachment_id, namespace_id, blob_sha256)blob_storage_attachments(id, namespace_id, sha256) and (namespace_id, blob_sha256)blob_storage_blobs(namespace_id, sha256).
  • file_name text NOT NULL with the char_length >= 1 AND <= 255 bound and a no-traversal CHECK rejecting /, \, and the bare . and .. traversal roots: a Maven file name is a leaf name, so a path separator or a bare traversal root is never valid in one.
  • upstream_etag text nullable, char_length <= 255, plus a CHECK rejecting every ASCII control character and DEL so a crafted upstream value can never be reflected verbatim into an outbound If-None-Match header. RFC 7232 admits only %x21 / %x23-7E / obs-text in an ETag, so the CHECK rejects no legitimate value.
  • upstream_checked_at timestamptz NOT NULL DEFAULT now(); soft_deleted_at nullable.

Six indexes:

Index Columns Purpose
Unique (namespace_id, maven_remote_version_id, file_name) WHERE soft_deleted_at IS NULL AND maven_remote_version_id IS NOT NULL Version-level cached-file lookup; admits re-caching after soft delete
Unique (namespace_id, maven_remote_package_id, file_name) WHERE soft_deleted_at IS NULL AND maven_remote_version_id IS NULL Package-level cached-file lookup; complementary to the version-level unique on the version column
Index (namespace_id, maven_remote_package_id) Full, non-partial FK coverage for the package parent
Index (namespace_id, maven_remote_version_id) Full, non-partial FK coverage for the version parent
Index (namespace_id, blob_storage_attachment_id) Cleanup joins from an attachment to its cached file rows
Index (namespace_id, blob_sha256) Reverse lookup from a stored blob sha256 (checksum search)

All six come from ADR-007 and the merged S14 data model, the non-partial FK-coverage pair being the spec's called-out divergence: the partial uniques exclude soft-deleted rows, so the RI check on a parent delete cannot use them.

Tests

maven_remote_files_schema_integration_test.go (28 test functions) asserts the post-Up shape and every constraint's accept and reject paths:

  • both composite blob FKs accepting present parents and rejecting absent ones, with fixtures seeding the parent blob_storage_blobs and blob_storage_attachments rows — the blob_storage_blobs FK's reject half is a catalog assertion because the attachment FK transitively requires the blob row, which the test comment records;
  • digest octet-length CHECKs rejecting wrong widths and accepting md5 NULL;
  • the upstream_etag CHECK rejecting CR, LF, CRLF, tab, VT, BEL, ESC, and DEL while accepting a valid ETag and NULL, and the file_name CHECK rejecting / and \ while accepting ordinary leaf names;
  • version-level and package-level rows coexisting for one file_name under the two partial uniques, each rejecting its own duplicate;
  • partition routing by HASH(namespace_id) and the per-index catalog shapes.

Step 1's static Down-section assertions (DownLockBudget, DownReversesEveryUpObject) run over the new migration through the mavenRemoteCoreMigrationTokens append, with the Down style pinned in mavenRemoteDownStyle as dropWithoutDetach.

MR size

~1,880 lines of reviewable code (407-line migration + 1,430-line test + ~45 lines across the existing test files and the plan), over the 500 LOC ceiling in the development model, which asks for a split or a justification here.

Justification: as in Steps 1–3, roughly 260 of the migration's 407 lines are the mechanical 64-partition CREATE DDL and its DROP reverse, reviewable as a block once the first is read. The generated jet types and the structure.sql dump are excluded as generated artifacts. The novel surface is the table body, its CHECKs and FKs, its six indexes, and the test suite. Splitting the partition DDL from the table it partitions would produce a non-applying migration, so the step is already at its minimum reviewable size.

End-to-end scenario catalogs

No scenario added or invalidated. docs/testing/ holds no Maven catalog yet — authoring the first one is a separate docs concern — and this step ships no request path, so there is no observable behavior to cover. Step 18's hermetic proxy harness is the automated coverage for the S14 read paths.

Conformance

Not applicable: this step implements no Maven protocol behavior. Conformance runs against the read paths landing in Steps 11 and 14 through 16.

Notes for reviewers

  • The file_name and upstream_etag CHECKs harden maven_remote_files beyond ADR-007. The upstream_etag CHECK rejects every ASCII control character and DEL (RFC 7232 admits only %x21 / %x23-7E / obs-text in an ETag, so the CHECK rejects no legitimate value), and the file_name CHECK rejects path separators (/, \) and the bare . and .. traversal roots. NUL is omitted from both because PostgreSQL text cannot contain 0x00 — the INSERT fails before any CHECK runs. The backslash is written chr(92) rather than a '\' literal because pg_format misreads the escaped-quote form and silently eats every second partition block from structure.sql. The spec amendment records these and the version-package composite FK as divergences pending an ADR-007 amendment, moving the count from seven to eight.
  • The migration declares -- +goose NO TRANSACTION for the same reason as Steps 1–3: a single transaction would hold every partition lock until commit, and the per-statement layout releases them as it goes so each statement stays within max_locks_per_transaction. Unlike Steps 1–3, this Down is re-runnable after a mid-sequence interruption: every DROP carries IF EXISTS and partitions precede the parent, so the accepted cost is the absence of atomicity, not replayability — the migration header documents this.
  • The Down shape differs from Steps 1–3 on purpose. This is the slice's only file-bearing table, so it takes the cache-content Down (DROP per partition, no DETACHDROP TABLE on a partition auto-detaches it and locks only that partition plus the parent). The style is pinned per migration in mavenRemoteDownStyle, and an unknown token fails the helper's membership check rather than silently passing.
  • The plan amendment corrects Step 4's Files entry. An earlier branch commit named schema_helpers_test.go as modified and described the Down-shape helper learning the cache-content shape here; that helper already accepts both shapes on main, and the pinning landed as mavenRemoteDownStyle in maven_remote_repositories_schema_integration_test.go. The entry now says so.
  • The migration was re-stamped mid-review. main's head advanced to 20260804120000 (create_container_remote_repositories) while this branch was in flight, so lint:migration-ordering rejected the original 20260801120000 stamp. The migration is now 20260805120000 and migrations_checksum_test.go moves knownHeadVersion to match. The rename is content-identical; no tracked reference named the old timestamp (migration tokens are timestamp-free substrings, and structure.sql carries no version rows).
  • The branch history records the review-feedback fixes from earlier rounds: the etag length correction, the column ordering by type size, the sprint-id sweep in comments, and the validate-step findings (a step reference in a rewritten comment, tests/tc table naming).

Database Review Evidence

Migrations

Note

Timings are from CI (db:migrate matrix, goose verbose) against an empty database, in apply / rollback order per PG version. Production-scale validation via Database Lab is not yet available. See Database review evidence for the matrix rationale and how to read the numbers.

Migration PG 16 PG 17 PG 18
20260805120000_create_maven_remote_files.sql OK (2.61s / 812.76ms) OK (1.35s / 804.67ms) OK (1.31s / 798.34ms)

Migration notes:

  • Up/down asymmetry is inverted versus Steps 1–3: rollback is faster than apply (798-813ms down against 1.31-2.61s up). Expected for this table's cache-content Down — each DROP auto-detaches its partition, where Steps 1–3 pay a separate DETACH pass per partition (6.27s-6.94s down). The Up direction — the boot-relevant phase — peaks at 2.61s, well within the 5-minute boot budget.
  • PG 16 apply (2.61s) is ~1.9x the PG 17/18 applies (1.35s and 1.31s). Steps 1–3 show the same ratio in their own jobs, so this reads as PG 16 runner variance rather than anything migration-specific.

Related to #286 (closed)

Edited by Moaz Khalifa

Merge request reports

Loading
Loading