feat(storage): add blob_storage_blobs and upload_sessions tables

What

Implements Step 6 of the Storage Layer (S06) plan: the two SQL schema tables and the deferred foreign key, per the S06 spec "Data Model".

  • 20260604120000_create_blob_storage_blobs.sql — content-addressable blob metadata. HASH(sha256) × 64 partitions, PK (id, namespace_id, sha256), unique dedup index (namespace_id, sha256), created_at for the GC grace period, nullable metadata_sha1 with a 20-byte CHECK, object_storage_key bounded to 1024 bytes, explicit id sequence (ADR-022), no updated_at (rows are immutable). FKs to namespaces/repositories deferred per the plan.
  • 20260604120100_create_upload_sessions.sql — in-flight upload sessions. HASH(namespace_id) × 64 partitions, PK (id, namespace_id), unique (namespace_id, upload_id), indexes on (expires_at) and (namespace_id, repository_id), size_bytes >= 0 CHECK, dirty poison-pill, hash_state BYTEA, explicit id sequence, no sha256 column (dropped from ADR-007).
  • 20260604120200_add_blob_storage_attachments_blob_fk.sql — the deferred fk_blob_storage_attachments_ns_id_and_sha256_blob_storage_blobs.
  • jet codegen for both tables and the regenerated structure.sql.

Deviation from the plan (FK migration)

The plan specified ADD CONSTRAINT ... NOT VALID + a separate VALIDATE CONSTRAINT. PostgreSQL rejects NOT VALID foreign keys on partitioned tables (SQLSTATE 42809), so the FK is added validating in a single statement. The foundation-only window keeps blob_storage_attachments empty (no INSERT callsites until the format Step 1s land), so the validation scan is trivially fast — the same safety rationale the plan cited for the cheap VALIDATE. Documented inline in the migration.

S12 FK fallout (test-only)

The new blob_storage_attachments → blob_storage_blobs FK requires a parent blob row for every attachment. Pre-existing S12 (OCI) integration tests seeded attachments without one, so each affected fixture now seeds a parent blob_storage_blobs row: container_blob, container_manifest, container_manifest_relationship, container_tag, container_blob_linker, the OCI upload-finalize integration tests, and the migrations-package OCI constraint test. The seed/cleanup helpers thread the caller's context (satisfies contextcheck) and order cleanups so a child attachment is removed before its parent blob (no ON DELETE CASCADE).

Tests

Schema-shape + FK integration tests were authored test-first (the test(storage) commit). internal/datastore, internal/datastore/migrations, and internal/format/oci all pass; mise run db:lint (squawk) is clean; mise run db:jet-gen + db:dump-structure regen is idempotent.

Spec coverage (schema, from the spec Data Model + plan Step 6 acceptance):

# Criterion Tests
DM-1 blob_storage_blobs columns/types; no updated_at TestBlobStorageBlobs_Columns
DM-2 PK (id, namespace_id, sha256) TestBlobStorageBlobs_PrimaryKey
DM-3 HASH(sha256) × 64 partitions TestBlobStorageBlobs_Partitions, …PartitionedByHashOfSHA256
DM-4 unique (namespace_id, sha256) dedup index TestBlobStorageBlobs_UniqueNamespaceSHA256Index
DM-5 metadata_sha1 CHECK (NULL or 20 bytes) …MetadataSHA1LengthCheckExists, …Rejects19ByteValue, …AllowsNull
DM-6 id explicit sequence TestBlobStorageBlobs_SequenceShape
DM-7 upload_sessions columns/types; hash_state, dirty, updated_at; no sha256 TestUploadSessions_Columns
DM-8 PK (id, namespace_id) TestUploadSessions_PrimaryKey
DM-9 HASH(namespace_id) × 64 partitions TestUploadSessions_Partitions, …PartitionedByHashOfNamespaceID
DM-10 three indexes: unique (ns, upload_id), (expires_at), (ns, repository_id) TestUploadSessions_ThreeNamedIndexes
DM-11 size_bytes >= 0 CHECK TestUploadSessions_SizeBytesNonNegativeCheck
DM-12 id explicit sequence TestUploadSessions_SequenceShape
DM-13 attachments→blobs FK; absent-sha256 INSERT fails SQLSTATE 23503 TestBlobStorageAttachmentStore_Create (blob-FK subtests)

Behavioral acceptance (PathResolver, StorageDriver, BlobStore, Session, middlewares) is owned by later S06 steps and is not in this MR.

References

  • Plan: docs/plans/2026-05-15-storage-layer.md — Step 6.
  • Spec: docs/specs/S06-storage-layer.md — Data Model.
  • ADR-007 (schema), ADR-008 (content-addressable storage), ADR-022 (cross-deployment migration).

Related to #155 (closed)

Database Review Evidence

Migrations

Note

Timings are from CI (db:migrate matrix, goose verbose) against an empty database, in apply / rollback order per PG version. Production-scale validation via Database Lab is not yet available. See Database review evidence for the matrix rationale and how to read the numbers.

Migration PG 16 PG 17 PG 18
20260612130000_create_blob_storage_blobs.sql OK (613.97ms / 131.39ms) OK (288.82ms / 143.27ms) OK (315.93ms / 175.12ms)
20260612130100_create_upload_sessions.sql OK (920.47ms / 148.19ms) OK (348.2ms / 167.81ms) OK (383.97ms / 213.57ms)
20260612130200_add_blob_storage_attachments_blob_fk.sql OK (172.13ms / 53.65ms) OK (256.41ms / 59.11ms) OK (188.46ms / 56.01ms)

Migration notes:

  • create_upload_sessions.sql: PG 16 apply (920ms) is ~2.4x PG 17 (348ms) and PG 18 (384ms). create_blob_storage_blobs.sql shows the same pattern more mildly (PG 16 614ms, ~1.9x). Both tables create 64 HASH partitions; the gap is PG 16 partitioned-DDL overhead, not a correctness issue. All timings are sub-second on an empty DB, well within the 5-minute per-migration boot budget, and rollbacks are fast and symmetric. Known and monitor-only; no action for this MR.
Edited by Pawel Rozlach

Merge request reports

Loading
Loading