feat(oci): container_remote_blobs schema (S16 Step 2a)

What

Adds the container_remote_blobs table — a cached blob under a remote-image entry — as the first of Step 2's three MRs in the S16 container remote vertical slice. One goose migration, the regenerated jet types and structure.sql dump, and two integration suites asserting the schema shape and every constraint's accept and reject paths.

No Go production code and no behavior change: nothing reads or writes this table until the cache-fill and cache-hit serve steps later in S16.

That does not make it cheap to undo. The parent DROP in the Down path takes ACCESS EXCLUSIVE on namespaces, container_remote_images, blob_storage_attachments, blob_storage_blobs and the 64 partitions of each of the latter three, and ACCESS EXCLUSIVE blocks reads as well as writes. The 64 partition DROPs that precede it take no lock on any FK target (see the Down comment in the migration), so plan a rollback as a single brief stall across the blob-storage layer, not a no-op.

Step 1 (container_remote_repositories, container_remote_images, container_remote_manifests) is merged. This MR is scoped to container_remote_blobs alone.

Spec: docs/specs/S16-container-remote.md#container_remote_blobs. Plan: docs/plans/2026-07-30-container-remote.md (Step 2, sub-MR 2a).

Schema

PARTITION BY HASH (namespace_id) × 64, PK (id, namespace_id), a composite FK to container_remote_images(id, namespace_id), two composite FKs into blob_storage_attachments/blob_storage_blobs (content-addressed storage), and an FK to namespaces — all ON DELETE NO ACTION. Two octet_length(...) = 32 CHECKs on digest and blob_sha256. Four indexes: a partial unique digest index (WHERE soft_deleted_at IS NULL), a non-partial FK-coverage index on (namespace_id, container_remote_image_id), one on (namespace_id, blob_storage_attachment_id), and the (namespace_id, blob_sha256) reverse lookup. Every column, constraint, and index decision is documented at the statement it governs in the migration.

Three columns container_remote_manifests carries are deliberately absent here, and each absence is a rule rather than an oversight — media_type, last_downloaded_at, and size. Each has its own attributable test rather than resting only on the exact-column-count assertion.

Two of these are corrections to ADR-007 as published, which declares media_type on this table and omits the created_at this migration adds. The spec calls out both. The corrections are argued in the migration's comments rather than by editing the ADR, which is synced from the handbook repo and is read-only here.

Merge order

!1396 (merged) merged first and this branch has rebased onto it. Both edit the plan's dependency-summary table on adjacent rows — !1396 (merged) rewrites rows 1-6 and 9, this branch rewrites rows 7 and 8 — so whichever landed second had to rebase. !1396 (merged) was the smaller docs-only change and went first. This branch now sits on main with !1396 (merged)'s re-baselined cells kept and rows 7 and 8 re-applied on top.

Within Step 2 itself the order is 2a → 2c → 2b, and the three migrations must be authored in that order too, not merely merged in it. A goose timestamp is fixed when the file is authored and internal/datastore/migrations/migrations.go sets goose.WithAllowOutofOrder(false), so a 2b authored ahead of 2c would be refused at deploy time by any database that had already applied 2c. This MR makes that explicit in the plan; the table rows there now read in merge order rather than alphabetically.

Review follow-ups applied

A local /review-branch pass produced three fixes beyond the plan edit above.

The Down path's comments described themselves as "replayable" and "re-runnable" after an interruption. Both are true only of re-running Down. goose deletes the goose_db_version row only once every statement in the section has succeeded, and the file runs under NO TRANSACTION, so an interrupted Down leaves the migration recorded as applied with some partitions already dropped. Up then skips the file rather than rebuilding them, and the gap stays invisible until the first insert whose namespace_id hashes into a dropped remainder fails. The comments now say which one to re-run and why. Verified against goose v3.27.3.

Seven merged migrations carry the same wording and are not corrected here, to keep this MR to its own table. Tracked in #568, which lists all seven, names the target wording, and records two things the fix needs that are not obvious: one Allow-Migration-Edit trailer per file, because lint:migration-immutability blocks comment-only edits to merged migrations just as it blocks DDL rewrites, and a comment-marker-stripped grep to find them all, because in two of the seven the phrase wraps across a -- continuation so a plain grep finds only five. The three Maven remote migrations that also use the word "replayable" are outside that scope: they say a partway Down is not replayable, which is accurate and needs no change.

TestContainerRemoteBlobsSchema_NoSizeColumn is new. The constraints suite's header claimed the missing media_type and size were both asserted directly; only media_type was. size has the strongest pull of the three absences — container_remote_manifests carries one with a NOT NULL and a non-negative CHECK, and this suite is shorter than the manifests one by exactly those two cases. Mutation-checked: adding a size column turns the new test red with must carry NO size column and the count assertion red with a want-8-got-9 listing.

explainKeysetPlan now disables enable_bitmapscan alongside enable_seqscan. The blob_sha256 pruning test is the only caller whose query has no ORDER BY, so it was the only one resting on the cost estimate rather than the query shape to rule out a bitmap plan — and assertIndexBackedKeysetScan rejects a bitmap node, so a planner difference across the 16/17/18 matrix would have surfaced as "the migration's index is missing". All six callers pass with it. Widening the accepted node prefixes was the alternative and is worse: it would let a bitmap plan through on the five callers where the ordered index scan is the property under test.

Database Review Evidence

Collected from pipeline 2750177783 on PostgreSQL 17, all three matrix jobs green:

Phase Duration
Apply 636.5 ms
Rollback 566.7 ms
Re-apply 505.6 ms

Full output is in the migration-review.log artifact of jobs 15828818237, 15828818238, and 15828818239 — not the job trace, which truncates at 4 MB and cuts off before the Down and Re-Up sections.

The table and its 64 partitions are created empty, so no phase touches existing rows. The lock exposure, which the timings do not show, is the one in ## What above.

The (namespace_id, blob_sha256) reverse-lookup index carries EXPLAIN (ANALYZE, BUFFERS) evidence in the schema suite: 5,000 seeded rows, a soft-deleted row as the search target, asserting the plan rides that index's partition child and prunes to one partition. The other three indexes have no such evidence, because no store or query-layer code queries this table yet. That evidence belongs to the MR that adds the first real query against it.

On the size of this MR

This MR is several times the 500 LOC guideline in docs/dev/development-model.md, matching the shape of the merged Step 1 MRs.

Part LOC
Migration SQL 530
— of which mechanical 64-partition DDL 193
Integration suites (constraints, schema, fixtures, shared helpers) ~1,870
Plan 54
knownHeadVersion and migration-token bump 4
Reviewable total ~2,450

structure.sql (+2,470) and the three go-jet files (+125) are generated and excluded from that count.

So the novel hand-written DDL is a few hundred lines once the partition statements come out (CONCURRENTLY is unavailable on a partitioned parent, and batching them into a DO $$ ... LOOP would accumulate all 65 locks in one transaction, which the per-statement layout avoids). The bulk is the two integration suites.

Splitting further would mean splitting one table's schema from its own tests. The plan's shape is one table per MR, and this is that table.

Suggested reading order

The commits are ordered for review and the split is meaningful:

  1. test(oci): container_remote_blobs schema tests — the suites, written first
  2. feat(oci): container_remote_blobs schema — the migration they constrain
  3. refactor: simplify container remote blob tests — code-simplifier pass
  4. the two docs(plans): commits — Step 2 merge order and the Files entry
  5. test(oci): pin container_remote_blobs FK match type and parents
  6. the final three — the /review-branch fixes described above

End-to-end scenario catalogs

No scenario added or invalidated. This step ships schema only: nothing reads or writes container_remote_blobs until the cache-fill and cache-hit serve steps land, so there is no observable client behavior for a scenario to cover. Both container catalogs (docs/testing/e2e/oci.md, docs/testing/e2e/docker.md) already list remote repositories as out of scope until the read paths ship.

Related to #288

Edited by Radamanthus Batnag

Merge request reports

Loading
Loading