fix(handlers): tolerate manifest subject not present on push
What
Stop rejecting an OCI manifest whose subject is not (yet) present in the repository.
At registry/handlers/manifests.go, when a pushed manifest carries a subject, the registry
looked up the subject manifest and returned MANIFEST_BLOB_UNKNOWN if it was absent. This
violates the OCI distribution spec (## Requirements -> ### Push -> #### Pushing Manifests,
https://github.com/opencontainers/distribution-spec/blob/main/spec.md#pushing-manifests):
A registry MUST initially accept an otherwise valid manifest with a
subjectfield that references a manifest that does not exist in the repository, allowing clients to push a manifest and referrers to that manifest in either order.
The same section still permits rejecting other absent references (config, layers, manifest-list children), so those checks are unchanged.
Why
docker buildx build --attest type=sbom --push produces an OCI index with an image manifest
and an SBOM attestation manifest whose subject is the image manifest. buildkit pushes them
concurrently and does not guarantee the image manifest lands first. When the attestation is
validated before the image manifest commits, the push fails with
blob unknown to registry - <image manifest digest>; a retry succeeds. This has caused
intermittent CI push failures.
Reproduced deterministically against a GitLab registry (DB enabled): pushing an OCI manifest
whose only issue is a subject pointing at an absent manifest returns
400 MANIFEST_BLOB_UNKNOWN, while the identical manifest without a subject returns
201 Created.
How
When the subject manifest is not found, store the manifest with a null subject_id and log a
warning instead of erroring. Gated by a new feature flag
REGISTRY_FF_TOLERATE_MISSING_SUBJECT (default off) for safe rollout.
This is the only push-time enforcement of subject existence: the OCI/schema2 validators
exclude the subject field, and the manifest-list path has no subject check.
Follow-up (not in this MR)
The referrers listing joins on subject_id, so a null subject_id omits the referrer. To
keep the referrers API complete regardless of push order, add a manifests.subject_digest
column populated on every push and backfill subject_id when the subject manifest is later
created. With backfill, the existing referrers query needs no change.
Testing
go build ./internal/feature/... ./registry/handlers/...passes.- Deterministic reproduction confirms the current 400 and the control 201; with the flag enabled the manifest is stored instead of rejected.
Related to #2375