feat(oci): container_remote_blobs schema (S16 Step 2a)
What
Adds the container_remote_blobs table — a cached blob under a remote-image
entry — as the first of Step 2's three MRs in the S16 container remote vertical
slice. One goose migration, the regenerated jet types and structure.sql dump,
and two integration suites asserting the schema shape and every constraint's
accept and reject paths.
No Go production code and no behavior change: nothing reads or writes this table until the cache-fill and cache-hit serve steps later in S16.
That does not make it cheap to undo. The parent DROP in the Down path takes
ACCESS EXCLUSIVE on namespaces, container_remote_images,
blob_storage_attachments, blob_storage_blobs and the 64 partitions of each of
the latter three, and ACCESS EXCLUSIVE blocks reads as well as writes. The 64
partition DROPs that precede it take no lock on any FK target (see the Down
comment in the migration), so plan a rollback as a single brief stall across the
blob-storage layer, not a no-op.
Step 1 (container_remote_repositories, container_remote_images,
container_remote_manifests) is merged. This MR is scoped to
container_remote_blobs alone.
Spec: docs/specs/S16-container-remote.md#container_remote_blobs.
Plan: docs/plans/2026-07-30-container-remote.md (Step 2, sub-MR 2a).
Schema
PARTITION BY HASH (namespace_id) × 64, PK (id, namespace_id), a composite FK
to container_remote_images(id, namespace_id), two composite FKs into
blob_storage_attachments/blob_storage_blobs (content-addressed storage), and
an FK to namespaces — all ON DELETE NO ACTION. Two octet_length(...) = 32
CHECKs on digest and blob_sha256. Four indexes: a partial unique digest index
(WHERE soft_deleted_at IS NULL), a non-partial FK-coverage index on
(namespace_id, container_remote_image_id), one on
(namespace_id, blob_storage_attachment_id), and the
(namespace_id, blob_sha256) reverse lookup. Every column, constraint, and index
decision is documented at the statement it governs in the migration.
Three columns container_remote_manifests carries are deliberately absent here,
and each absence is a rule rather than an oversight — media_type,
last_downloaded_at, and size. Each has its own attributable test rather than
resting only on the exact-column-count assertion.
Two of these are corrections to ADR-007 as published, which declares media_type
on this table and omits the created_at this migration adds. The spec calls out
both. The corrections are argued in the migration's comments rather than by
editing the ADR, which is synced from the handbook repo and is read-only here.
Merge order
!1396 (merged) merged first and this branch has rebased onto it. Both edit the plan's
dependency-summary table on adjacent rows — !1396 (merged) rewrites rows 1-6 and 9, this
branch rewrites rows 7 and 8 — so whichever landed second had to rebase. !1396 (merged)
was the smaller docs-only change and went first. This branch now sits on main
with !1396 (merged)'s re-baselined cells kept and rows 7 and 8 re-applied on top.
Within Step 2 itself the order is 2a → 2c → 2b, and the three migrations must
be authored in that order too, not merely merged in it. A goose timestamp is
fixed when the file is authored and
internal/datastore/migrations/migrations.go sets
goose.WithAllowOutofOrder(false), so a 2b authored ahead of 2c would be refused
at deploy time by any database that had already applied 2c. This MR makes that
explicit in the plan; the table rows there now read in merge order rather than
alphabetically.
Review follow-ups applied
A local /review-branch pass produced three fixes beyond the plan edit above.
The Down path's comments described themselves as "replayable" and "re-runnable"
after an interruption. Both are true only of re-running Down. goose deletes
the goose_db_version row only once every statement in the section has
succeeded, and the file runs under NO TRANSACTION, so an interrupted Down
leaves the migration recorded as applied with some partitions already dropped.
Up then skips the file rather than rebuilding them, and the gap stays invisible
until the first insert whose namespace_id hashes into a dropped remainder
fails. The comments now say which one to re-run and why. Verified against goose
v3.27.3.
Seven merged migrations carry the same wording and are not corrected here, to
keep this MR to its own table. Tracked in
#568,
which lists all seven, names the target wording, and records two things the fix
needs that are not obvious: one Allow-Migration-Edit trailer per file, because
lint:migration-immutability blocks comment-only edits to merged migrations just
as it blocks DDL rewrites, and a comment-marker-stripped grep to find them all,
because in two of the seven the phrase wraps across a -- continuation so a
plain grep finds only five. The three Maven remote migrations that also use the
word "replayable" are outside that scope: they say a partway Down is not
replayable, which is accurate and needs no change.
TestContainerRemoteBlobsSchema_NoSizeColumn is new. The constraints suite's
header claimed the missing media_type and size were both asserted directly;
only media_type was. size has the strongest pull of the three absences —
container_remote_manifests carries one with a NOT NULL and a non-negative
CHECK, and this suite is shorter than the manifests one by exactly those two
cases. Mutation-checked: adding a size column turns the new test red with
must carry NO size column and the count assertion red with a want-8-got-9
listing.
explainKeysetPlan now disables enable_bitmapscan alongside enable_seqscan.
The blob_sha256 pruning test is the only caller whose query has no ORDER BY,
so it was the only one resting on the cost estimate rather than the query shape
to rule out a bitmap plan — and assertIndexBackedKeysetScan rejects a bitmap
node, so a planner difference across the 16/17/18 matrix would have surfaced as
"the migration's index is missing". All six callers pass with it. Widening the
accepted node prefixes was the alternative and is worse: it would let a bitmap
plan through on the five callers where the ordered index scan is the property
under test.
Database Review Evidence
Collected from pipeline 2750177783 on PostgreSQL 17, all three matrix jobs green:
| Phase | Duration |
|---|---|
| Apply | 636.5 ms |
| Rollback | 566.7 ms |
| Re-apply | 505.6 ms |
Full output is in the migration-review.log artifact of jobs 15828818237,
15828818238, and 15828818239 — not the job trace, which truncates at 4 MB and
cuts off before the Down and Re-Up sections.
The table and its 64 partitions are created empty, so no phase touches existing
rows. The lock exposure, which the timings do not show, is the one in ## What
above.
The (namespace_id, blob_sha256) reverse-lookup index carries
EXPLAIN (ANALYZE, BUFFERS) evidence in the schema suite: 5,000 seeded rows, a
soft-deleted row as the search target, asserting the plan rides that index's
partition child and prunes to one partition. The other three indexes have no
such evidence, because no store or query-layer code queries this table yet. That
evidence belongs to the MR that adds the first real query against it.
On the size of this MR
This MR is several times the 500 LOC guideline in
docs/dev/development-model.md, matching the shape of the merged Step 1 MRs.
| Part | LOC |
|---|---|
| Migration SQL | 530 |
| — of which mechanical 64-partition DDL | 193 |
| Integration suites (constraints, schema, fixtures, shared helpers) | ~1,870 |
| Plan | 54 |
knownHeadVersion and migration-token bump |
4 |
| Reviewable total | ~2,450 |
structure.sql (+2,470) and the three go-jet files (+125) are generated and
excluded from that count.
So the novel hand-written DDL is a few hundred lines once the partition
statements come out (CONCURRENTLY is unavailable on a partitioned parent, and
batching them into a DO $$ ... LOOP would accumulate all 65 locks in one
transaction, which the per-statement layout avoids). The bulk is the two
integration suites.
Splitting further would mean splitting one table's schema from its own tests. The plan's shape is one table per MR, and this is that table.
Suggested reading order
The commits are ordered for review and the split is meaningful:
test(oci): container_remote_blobs schema tests— the suites, written firstfeat(oci): container_remote_blobs schema— the migration they constrainrefactor: simplify container remote blob tests— code-simplifier pass- the two
docs(plans):commits — Step 2 merge order and the Files entry test(oci): pin container_remote_blobs FK match type and parents- the final three — the
/review-branchfixes described above
End-to-end scenario catalogs
No scenario added or invalidated. This step ships schema only: nothing reads or
writes container_remote_blobs until the cache-fill and cache-hit serve steps
land, so there is no observable client behavior for a scenario to cover. Both
container catalogs (docs/testing/e2e/oci.md, docs/testing/e2e/docker.md)
already list remote repositories as out of scope until the read paths ship.
Related to #288