feat(oci): blob upload PUT, GET, DELETE finalize (S12 Step 9)

Why

Step 9 of the OCI-local plan (S12). Step 8 (!347 (merged), merged) left the blob-upload surface half-open: a session could be initiated and appended to, but never finalized. This MR adds PUT blobs/uploads/<id> (finalize), single-shot POST blobs/uploads/?digest=, GET (status), and DELETE (cancel), so a client can push a blob end to end.

Also folded in (pre-existing, not Step 9): the OCI bearer-token endpoint was never mounted. The auth middleware advertised /v2/auth/token as the Bearer-challenge realm, but no handler was wired there, so the path 404'd through the /v2/ dispatcher and every authenticated request retried with an empty token and 401'd. server.New now mounts the handler via auth.RegisterTokenEndpoint, over the same validator the challenge middleware uses, which takes conformance:oci from 2/80 to 24/80. The remaining 51 failures are the still-incomplete OCI surface (manifest push lands in Step 12, content discovery and management after), not auth. Folded in here because it unblocks the blob-push surface this MR completes.

Stacked on !362 (merged) (S04 Step 3, db composition root). It targets sahmed/db-composition-root-step3 because the finalize integration test consumes that branch's datastore and composition-root wiring. GitLab retargets this MR to main when !362 (merged) merges. Draft until then, since it cannot merge to main before its base.

What (non-obvious)

  • The finalize transaction lives in the datastore, not the format package. datastore.ContainerBlobLinker owns the *sql.Tx that inserts blob_storage_attachments then container_blobs. internal/format/oci consumes it through a consumer-side seam and never imports database/sql, per ADR-023's no-direct-db rule.
  • The attachment insert is intentionally non-idempotent. A re-push of the same digest leaves a fresh orphan blob_storage_attachments row (reclaimed by ADR-011), while container_blobs carries ON CONFLICT DO NOTHING. So a re-push keeps container_blobs at one row and grows blob_storage_attachments (S12 "Orphaned attachments").
  • container_image_id is re-resolved from the request URL at finalize via an idempotent upsert, not stored on the session (S12 Session-to-URL binding).
  • The blob size cap (50 GiB default) is enforced before staging via a Content-Length pre-check plus http.MaxBytesReader against an under-declared body. Over-limit returns 413 SIZE_INVALID.

Test plan

  • go test ./internal/format/oci/ ./internal/datastore/ — unit suite: handler PUT/GET/DELETE/single-POST, the rapid full-lifecycle state machine, and the chunked-reassembly property.
  • go test -tags=integration -run TestCompleteUploadInsertsAttachmentAndBlob ./internal/format/oci/ — real DB via testcontainers; asserts one container_blobs row and two blob_storage_attachments rows after a re-push.
  • conformance:oci runs in CI (allow_failure until Step 18); the blob-push path is now exercised end to end.
  • golangci-lint 2.12 clean.

Spec coverage

Spec: docs/specs/S12-container-oci-local.md

Properties (universally quantified invariants)

# Property Tests
P-2 Digest correctness TestUploadPUT_FinalizeReturns201, TestUploadPOST_SingleShotReturns201, TestStateMachineUploadSession_Full, TestCompleteUploadInsertsAttachmentAndBlob (int)
P-3 Chunked upload reassembly TestUploadPUT_FinalizeReturns201, TestUploadPUT_FinalizeWithFinalChunk, TestPropertyChunkedReassembly (Step 8, retained)

Acceptance criteria (scenarios)

# Scenario Tests
AC-1 Blob upload (monolithic) TestUploadPUT_FinalizeReturns201, TestUploadPUT_FinalizeWithFinalChunk, TestCompleteUploadInsertsAttachmentAndBlob (int)
AC-2 Blob upload (single POST) TestUploadPOST_SingleShotReturns201, TestUploadPOST_SingleShotDigestMismatchReturns400
AC-3 Blob upload (chunked) TestUploadPUT_FinalizeReturns201 (POST->PATCH->PUT), TestStateMachineUploadSession_Full, TestCompleteUploadInsertsAttachmentAndBlob (int)
AC-21 Zero-byte blob TestUploadPOST_SingleShotZeroByteBlob
AC-27 Upload session GET TestUploadGET_StatusReturns204WithRange, TestUploadGET_StatusFreshSessionOmitsRange, TestStateMachineUploadSession_Full
AC-28 Upload session DELETE TestUploadDELETE_CancelReturns204, TestUploadDELETE_CancelThenGetReturns404, TestStateMachineUploadSession_Full

Error cases

# Code HTTP Condition Tests
E-3 BLOB_UPLOAD_UNKNOWN 404 Upload session not found, expired, completed, cancelled TestUploadPUT_UnknownSessionReturns404, TestUploadPUT_NonUUIDSessionReturns404, TestUploadPUT_CrossRepositorySessionReturns404, TestUploadGET_UnknownSessionReturns404, TestUploadGET_CrossRepositorySessionReturns404, TestUploadDELETE_UnknownSessionReturns404, TestUploadDELETE_CrossRepositorySessionReturns404, TestStateMachineUploadSession_Full
E-12 SIZE_INVALID 413 Blob exceeds blob_max_size TestUploadPUT_FinalChunkExceedsLimitReturns413, TestUploadPOST_SingleShotExceedsLimitReturns413

Security considerations

# Concern Tests
S-13 Digest verification on all writes TestUploadPUT_FinalizeReturns201, TestUploadPUT_DigestMismatchReturns400, TestUploadPOST_SingleShotDigestMismatchReturns400, TestCompleteUploadInsertsAttachmentAndBlob (int)

Adjacent rows touched (owner is another step; exercised here)

# Concern / Code Owner step Tests
E-4 DIGEST_INVALID (400), PUT mismatch Step 5 TestUploadPUT_DigestMismatchReturns400, TestUploadPUT_MissingDigestReturns400, TestUploadPUT_MalformedDigestReturns400, TestUploadPOST_SingleShotDigestMismatchReturns400
S-3 Digest format / lowercase canonical Step 5 TestUploadPUT_MalformedDigestReturns400
S-8 Blob size limit (DoS) TestUploadPUT_FinalChunkExceedsLimitReturns413, TestUploadPOST_SingleShotExceedsLimitReturns413
E-18 INTERNAL (500), unexpected failure Step 5 TestUploadPUT_CommitStorageErrorReturns500, TestUploadPUT_LinkErrorReturns500, TestUploadDELETE_CancelErrorReturns500, TestUploadGET_StatusStorageErrorReturns500
Context for LLM reviewers

Design rationale.

  • The blob-link transaction was relocated from internal/format/oci/db_blob_linker.go (deleted) to internal/datastore/container_blob_linker.go so the format package no longer imports database/sql. ADR-023's no-direct-db depguard rule (Proposed, not yet in .golangci.yaml) fences database/sql out of internal/format/**. The datastore type satisfies the unexported oci.blobLinker seam at the NewUploadHandler call site, so oci stays decoupled from datastore (the seam's purpose); no var _ assertion is added inside oci. The integration test's raw-SQL seed and assertion helpers keep database/sql; depguard !$test exempts test files.
  • The plan's Step 9 Tests entry previously said the re-push "asserts no second insert in either table." That contradicted the spec's orphan-attachment behavior and the as-built code. Corrected in this MR (docs(plans) commit) to match S12 and the test.

Non-goals (deferred by the plan, not gaps in this MR).

  • GC delete-lock / reachability events / 503 UNAVAILABLE — S20.
  • Soft-delete and repositories.size_bytes accounting — S20 / S22.
  • The optional Content-Range-on-PUT validation sub-path (S12 PUT completion) — the plan scopes PUT here as monolithic single-shot; no implemented endpoint diverges from the spec shape.
  • hash_state cross-Go-release re-hash fallback (S12 OQ-6) — needs the real upload_sessions datastore plus S03.

Known follow-ups (validation WARNs, not addressed here): stale doc comments in upload.go describing pre-Step-9 state; the 50 GiB code default vs ADR-004's 50 GB literal (config-overridable).

Edited by Hayley Swimelar

Merge request reports

Loading
Loading