feat(oci): blob cross-repository mount (S12 Step 16)

Hayley Swimelarrequested to merge
hswimelar/oci-hosted-step-16 into main

Why

S12 Step 16: blob cross-repository mount (POST /v2/<name>/blobs/uploads/?mount=<digest>&from=<source>), completing the OCI blob upload surface. Mount lets a client link a blob that already exists in another repository into a new one without re-uploading the bytes (OCI Distribution Spec section 3.3.2), the deduplication path registries rely on for shared base layers.

Built parallel to Phase D (Steps 12-15) with operator sign-off: mount is a pure blob operation with no manifest dependency, so it branches off main ahead of the manifest steps.

What (non-obvious)

  • 201/202 only, no info leak (S-2). Mount returns 201 on success or 202 (fallback to a fresh upload session) for every failure: missing source, denied source, or an unparseable, cross-namespace, or absent from. It never returns 4xx, so a failed mount cannot reveal whether the source repo or blob exists. The six fallback conditions each stamp a distinct internal oci.mount.fallback_reason wide-event value, but the client-visible 202 is byte-identical across all six (a rapid property test proves it).
  • from is the full route path <slug>/container/<repo>/<image>, resolved scoped to the destination namespace. A cross-namespace from falls back (ADR-008).
  • Database-only (ADR-008). MountBlob links the source's existing CAS object into the destination via blob_storage_attachments + container_blobs. No payload is copied. Re-mount returns 201 with no second container_blobs row (its ON CONFLICT DO NOTHING plus a re-read). The attachment insert is not guarded, so a re-mount adds an orphan attachment row reclaimed by ADR-011.
  • Dual auth. Write-to-dest is the existing route auth. Source-read goes through a sourceAuthorizer seam wired allow-all until S08/ADR-020 (matching the current bootstrap-token stub); the deny path is exercised now via an injected authorizer.

Test plan

go test ./internal/format/oci/... (unit, rapid property, and DB-backed integration), go vet (default and integration tags), and golangci-lint 2.12 all pass.

Spec coverage (S12 Step 16):

Criterion Tests
AC-4 success links blob into dest TestMountSucceeds*, TestMountBlobLinksDestNamespace (integration)
AC-5 six 202 fallbacks, distinct reasons TestMountFallbackEmitsReason, TestMountFallbackIndistinguishable (rapid)
S-2 no source info leak TestMountSourceDeniedNeverLeaks
?mount= wins over ?digest= TestMountWinsOverDigest
mount-then-pull byte-identical TestMountThenPullReturnsSourceBytes (integration)

Conformance: the crossmount spec runs under conformance:oci, which stays allow_failure until Step 18 flips the S12 gate.

Related to #19 (closed)

Database Review Evidence

Queries

Note

Plans are from EXPLAIN (ANALYZE, BUFFERS) against an ephemeral PostgreSQL 17.10 container (matching GL_PG_CURR_VERSION from .gitlab-ci-other-versions.yml), with synthesized seed data rolled back per query and the container torn down at the end of the run. Numbers reflect moderate cardinality and do not capture production-scale effects. See Database review evidence for seed sizing, methodology, and the anomalies the skill flags. Expand the row's details for the seed shape, rendered SQL, bound args, and raw plan.

Method Plan node Index Rows (plan / actual) Cost Time Buffers (hit / read) Partitions
ContainerBlobStore.FindBlobInRepository Result (InitPlan → Nested Loop) index_container_blobs_on_namespace_id_and_digest 1 / 1 9.34 0.032ms 4 / 0 1 of 64 (per table)
ContainerBlobStore.FindBlobInRepository

Summary: The plan matches the method's intent. The EXISTS subquery probes index_container_blobs_on_namespace_id_and_digest for the (namespace_id, digest) pair, and the namespace_id parameter prunes both container_blobs and container_images to a single hash partition (1 of 64). The image side is a 1-row Seq Scan inside the pruned partition, which the planner correctly prefers over an index probe at that size. Estimates match actuals (1 / 1) and execution stays at 0.032ms against 5000 seeded blobs. The found=false path (digest absent) takes the same plan: the index probe returns zero rows and the join short-circuits (never executed) at 0.042ms. No anomalies. The statement is built with Jet; its rendered SQL is plan-equivalent to a hand-written SELECT EXISTS, and the two-column join keeps the partition prune on both tables.

Seed shape: namespaces=1, repositories=1, container_repositories=1, container_images=1, blob_storage_blobs=1, blob_storage_attachments=1, container_blobs=5000

Rendered SQL (built with Jet, findBlobInRepositoryExists):

SELECT EXISTS (
          SELECT $1
          FROM public.container_blobs
               INNER JOIN public.container_images ON ((container_blobs.container_image_id = container_images.id) AND (container_blobs.namespace_id = container_images.namespace_id))
          WHERE ((container_images.namespace_id = $2::uuid) AND (container_images.container_repository_id = $3)) AND (container_blobs.digest = $4::bytea)
     )

Bound args: [$1=1 (EXISTS inner projection), $2 namespace_id=c547ecff-403f-4c6f-a7f4-1902179f3a4a, $3 container_repository_id=3, $4 digest=\x…09c4 (a seeded digest, decode(lpad(to_hex(2500),64,'0'),'hex'))]

Plan (EXPLAIN (ANALYZE, BUFFERS), found=true — digest present):

 Result  (cost=9.33..9.34 rows=1 width=1) (actual time=0.014..0.014 rows=1 loops=1)
   Buffers: shared hit=4
   InitPlan 1
     ->  Nested Loop  (cost=0.28..9.33 rows=1 width=0) (actual time=0.013..0.013 rows=1 loops=1)
           Join Filter: (container_blobs.container_image_id = container_images.id)
           Buffers: shared hit=4
           ->  Index Scan using container_blobs_p12_namespace_id_digest_idx on container_blobs_p12 container_blobs  (cost=0.28..8.30 rows=1 width=24) (actual time=0.008..0.009 rows=1 loops=1)
                 Index Cond: ((namespace_id = 'c547ecff-403f-4c6f-a7f4-1902179f3a4a'::uuid) AND (digest = '\x00000000000000000000000000000000000000000000000000000000000009c4'::bytea))
                 Buffers: shared hit=3
           ->  Seq Scan on container_images_p12 container_images  (cost=0.00..1.01 rows=1 width=24) (actual time=0.003..0.003 rows=1 loops=1)
                 Filter: ((namespace_id = 'c547ecff-403f-4c6f-a7f4-1902179f3a4a'::uuid) AND (container_repository_id = '3'::bigint))
                 Buffers: shared hit=1
 Planning:
   Buffers: shared hit=248
 Planning Time: 0.932 ms
 Execution Time: 0.032 ms

Plan (found=false — digest absent, the 202 source_missing fallback path):

 Result  (cost=9.33..9.34 rows=1 width=1) (actual time=0.012..0.013 rows=1 loops=1)
   Buffers: shared hit=2
   InitPlan 1
     ->  Nested Loop  (cost=0.28..9.33 rows=1 width=0) (actual time=0.011..0.012 rows=0 loops=1)
           Join Filter: (container_blobs.container_image_id = container_images.id)
           Buffers: shared hit=2
           ->  Index Scan using container_blobs_p12_namespace_id_digest_idx on container_blobs_p12 container_blobs  (cost=0.28..8.30 rows=1 width=24) (actual time=0.011..0.011 rows=0 loops=1)
                 Index Cond: ((namespace_id = 'c547ecff-403f-4c6f-a7f4-1902179f3a4a'::uuid) AND (digest = '\x00000000000000000000000000000000000000000000000000000000000f423f'::bytea))
                 Buffers: shared hit=2
           ->  Seq Scan on container_images_p12 container_images  (cost=0.00..1.01 rows=1 width=24) (never executed)
                 Filter: ((namespace_id = 'c547ecff-403f-4c6f-a7f4-1902179f3a4a'::uuid) AND (container_repository_id = '3'::bigint))
 Planning Time: 0.159 ms
 Execution Time: 0.042 ms

Timings (found=true): planning 0.932ms, execution 0.032ms, total 0.964ms.

Query notes: No anomalies flagged. The selective predicate uses index_container_blobs_on_namespace_id_and_digest, the namespace_id parameter prunes to a single partition on both tables, and planner estimates match seeded actuals.

Context for LLM reviewers

Design rationale.

  • The 201/202-only contract is a security control (S-2), not a convenience. Returning 404 on a missing source or 403 on a denied source would leak the source repository's existence and the blob's presence. Every failure mode collapses to a byte-identical 202 plus a fresh upload session, with the distinguishing reason confined to the internal wide event.
  • Mount is database-only per ADR-008: it adds a reference (blob_storage_attachments + container_blobs) to the source's existing CAS object rather than copying bytes. Cross-namespace mount is unsupported per ADR-008 deduplication scope and falls back.
  • allowAllMountSourceAuthorizer is a deliberate S08/ADR-020 stand-in matching the current bootstrap-token "all scopes" posture, not a missing check. The sourceAuthorizer seam exists so the real per-repository read-scope check is a drop-in later, and the deny branch is tested now.

Two non-feat commits in this MR.

  • docs(oci): correct S12 Step 16 plan prose fixes three plan errors that diverged from the spec and the as-built code (the canonical from form, the idempotency mechanism, and the datastore method name) and adds the wiring and datastore files to the Files entry. The spec governs; the code already matched it.
  • refactor(oci): extract resolveMountSource pulls the from-parse / resolve / authorize sequence out of handleMount to satisfy the 70-line funlen limit (94 to 62 lines). Behavior-preserving.

Non-goals (deferred, not omissions).

  • Size accounting (repositories.size_bytes, artifact counts) on mount: S22, consistent with the finalize path, which also writes none.
  • Garbage collection and soft-delete: S20.
  • Real per-repository source read-scope check: S08/ADR-020 (allow-all stand-in now).
  • Phase D manifest steps 13-15: tracked separately; mount has no manifest dependency.
Edited by Hayley Swimelar

Merge request reports