feat(oci): container virtual coordinate and rule adapter (S32 plan: 7/19)
Adds the container side of S13's rule vocabulary: the defined target-field set,
the per-request addressed values, and the container_virtual_upstream_rules
row mapping onto []virtual.Rule.
VirtualTargetFields returns the two target fields container defines, in
target_field column order. It builds the slice per call, so no caller can
widen the set the format declares.
NewVirtualCoordinate validates the target before it decomposes it. The order
is the kind, then the tag's length, then the tag grammar, then the image-name
length, then the image grammar. The kind comes first because the tag check
depends on it: a tag-addressed read must carry a tag, and every other shape
must carry none. A length check precedes its grammar in both cases because it
is O(1) and caps the regex scan, which is the more expensive of the two. The
tag's gate also keeps a crafted tag out of its own rejection: that arm prints
the length, so the surviving %q renders at most 128 bytes of input. The cap
is tagGrammar's own 128, which
internal/managementapi/container_list.go already names as the tag's only
bound. Each rejection wraps ErrVirtualTarget and returns the zero
coordinate.
ErrVirtualTarget covers a caller bug and a malformed tag alike, so its doc
tells a caller to answer 404, not 500. The OCI read path applies no tag
format check on pull — readReferenceDigestInvalid says so — so a malformed
tag is client input, and 404 MANIFEST_UNKNOWN is what the hosted path gives
for the same reference.
VirtualRulesFrom maps stored rule values onto matcher rules, row for row, in
order. It filters nothing and validates nothing. A corrupt rule_type or
target_field has to reach the matcher, which fails its upstream closed. A
rule dropped before the matcher runs is a rule the matcher cannot reject.
The npm mirror is untouched
An earlier revision of this branch also edited
internal/format/npm/virtual_rules.go, on the reasoning that both adapters
carried the same doc omissions and a declared mirror should not drift. That is
reverted. This step's Files entry names two files, both Create, both
under internal/format/oci; it names the npm file only as the one the new file
mirrors, never as one to modify. A mirror justifies copying npm's wording into
container, not pushing container's wording back into npm.
Nothing is lost. main's npm doc already opens with "It drops nothing and
validates nothing", so the clause this branch briefly deleted was only ever
missing here. The two genuine improvements it also carried — a length line on
Pattern and a short-read clause on VirtualRulesFrom — belong to S31, whose
DRI owns that file, and move to a docs(npm) merge request of their own.
Diff size
1126 LOC, past the 500 reviewable-LOC line docs/dev/development-model.md draws. Split by file group:
| Group | LOC |
|---|---|
internal/format/oci/virtual_rules.go |
176 |
internal/format/oci/virtual_rules_test.go |
950 |
Splitting would not help. 950 of the 1126 are tests, and the plan's own sizing
anticipated that ratio: the npm equivalent shipped 118 source against 769 test
on the same acceptance criterion. The source is one validating constructor and
two mapping functions with no interior seam, so a split would separate the
table from the mapping it pins. The test bulk comes from two demands the plan
makes explicitly — each field's mapping asserted separately, because the map is
keyed by an untyped int and a transposed key compiles, and every corrupt-value
case widened across all three request shapes.
Conformance tests
Not applicable. This step adds no protocol surface: no route, no handler, no
header, no status code. NewVirtualCoordinate and VirtualRulesFrom are pure
functions over values the resolver already holds.
e2e scenario catalog
No scenario is added or affected. docs/testing/ catalogs request-level scenarios, and this step ships no request path. Plan Step 18 adds the container virtual entries once the handlers land.
Spec coverage
Spec: docs/specs/S32-container-virtual.md
Acceptance criteria
| # | Criterion | Tests |
|---|---|---|
| R-1 | Virtual manifest pull picks the priority winner | Plan Step 15. Not this step. |
| R-2 | Warm digest pull short-circuits | Plan Step 15. Not this step. |
| R-3 | Hosted Lookup reads the authoritative tables | Plan Step 5. Not this step. |
| R-4 | Uncached virtual blob pull | Plan Step 16. Not this step. |
| R-5 | Statement count follows upstream kinds, not position count | Plan Step 10b. Not this step. |
| R-6 | Content negotiation runs cold | Plan Step 15. Not this step. |
| R-7 | An index is served without consulting its children | Plan Step 15. Not this step. |
| R-8 | A child manifest resolves on its own | Plan Step 15. Not this step. |
| R-9 | An unresolvable child does not retroactively fail the index | Plan Step 15. Not this step. |
| R-10 | No winner with an absence established | Plan Step 11. Not this step. |
| R-11 | A mixed unknown-and-absent resolution still answers not-found | Plan Step 11. Not this step. |
| R-12 | A data error withholds the not-found even where an absence was established | Plan Steps 10a and 10b. Not this step. |
| R-13 | No winner, no absence, a cache entry exists | Plan Step 11. Not this step. |
| R-14 | No winner, no absence, no cache entry | Plan Step 11. Not this step. |
| R-15 | A rule-excluded position is barred from the fallback | Plan Step 10b. Not this step. |
| R-16 | No upstreams configured | Plan Step 11. Not this step. |
| R-17 | An all-soft-deleted list is answered the same way | Plan Step 10b. Not this step. |
| R-18 | All eligible upstreams unhealthy | Plan Step 10b. Not this step. |
| R-19 | A soft-deleted upstream is skipped | Plan Step 10b. Not this step. |
| R-20 | A soft-deleted upstream cannot supply the cache fallback | Plan Step 10b. Not this step. |
| R-21 | A cold HEAD fills, and reports the fetched resource |
Plan Steps 15 and 16. Not this step. |
| R-22 | A cold HEAD does not answer from the Probe |
Plan Steps 15 and 16. Not this step. |
| R-23 | A winner's non-2xx Fetch does not fall through | Plan Step 11. Not this step. |
| R-24 | A winner's transport failure falls to the cache, not to a lower position | Plan Step 11. Not this step. |
| R-25 | A winner's digest mismatch serves nothing in the winner's place | Plan Step 12. Not this step. |
| R-26 | A tag-addressed pull fails the same way on the upstream's own digest | Plan Step 12. Not this step. |
| R-27 | Retry-After carries the value of its row's class, and there are four classes |
Plan Step 11. Not this step. |
| R-28 | The veto 503 is the one whose value does not follow from its verdict |
Plan Step 11, and gated on the S13 amendment the spec names. Not this step. |
| P-1 | Probe carries a pull-scoped token | Plan Step 9. Not this step. |
| P-2 | Cache hit avoids re-exchange | Plan Step 9. Not this step. |
| P-3 | Cold Probes coalesce their handshake | Plan Step 9. Not this step. |
| P-4 | The two budgets are independent | Plan Step 9. Not this step. |
| P-5 | Probe outcomes map to phase results | Plan Step 9. Not this step. |
| P-6 | A Probe 401 on a cached token re-exchanges once |
Plan Step 9. Not this step. |
| P-7 | A slow first round-trip does not eat the retry's budget | Plan Step 9. Not this step. |
| AD-1 | Tag-targeted deny excludes a tag-addressed request | TestNewVirtualCoordinate_TagRules ("a tag deny rule matching the tag excludes the read", "a tag deny wildcard matching the tag excludes the read", "a tag deny rule naming another tag leaves the read eligible") |
| AD-2 | Tag-targeted allow keeps an upstream eligible for a matching tag | TestNewVirtualCoordinate_TagRules ("a tag allow rule matching the tag admits the read", "a tag allow wildcard admits a tag the pattern covers", "a tag allow rule naming another tag excludes the read", "a tag allow wildcard excludes a tag the pattern misses") |
| AD-3 | Tag-targeted deny does not exclude a digest request | TestNewVirtualCoordinate_TagRules ("a tag deny rule leaves a digest-addressed manifest read eligible", "a tag deny wildcard leaves a digest-addressed manifest read eligible", "a tag deny rule leaves a blob read eligible") |
| AD-4 | Tag-targeted allow does not exclude one either | TestNewVirtualCoordinate_TagRules ("a tag allow rule leaves a digest-addressed manifest read eligible", "a tag allow rule leaves a blob read eligible", "a tag allow rule naming a tag the read does not carry still leaves a blob read eligible") |
| AD-5 | Image-targeted rules govern both | TestNewVirtualCoordinate_ImageRules (the deny and allow rows across all three shapes) |
| AD-6 | A corrupt target_field fails closed on the requests that do not address it |
TestNewVirtualCoordinate_CorruptRuleValuesFailClosed (the digest-read and blob-read rows for an out-of-range and a negative field, and the corrupted-deny rows) |
| AD-7 | The slice filters nothing | TestVirtualRulesFrom ("a corrupt row is not dropped from a set of well-formed ones", "row order is preserved when the same field repeats", the out-of-range, negative, and unreadable rows); TestNewVirtualCoordinate_CorruptRuleValuesFailClosed ("a corrupt row outranks a matching allow rule beside it") |
| AD-8 | All denied answers not-found | Plan Step 11 (the resolution-level verdict). This step covers the per-upstream denial only. |
| AD-9 | A corrupt rule answers 500, even in a mixed set |
Plan Steps 10a and 10b. This step covers the per-upstream ineligibility only. |
| AD-10 | An upstream of an unexpected kind fails closed | Plan Step 10b. Not this step. |
| AD-11 | An upstream of the wrong format fails closed too | Plan Step 10b. Not this step. |
| MD-1 | Manifest write | Plan Step 13. Not this step. |
| MD-2 | Blob upload | Plan Step 13. Not this step. |
| MD-3 | Authorization precedes the kind dispatch | Plan Step 13. Not this step. |
| MD-4 | Virtual tag listing is not found | Plan Step 13. Not this step. |
| MD-5 | Virtual referrers is not found | Plan Step 13. Not this step. |
| C-1 | One upstream fetch across concurrent virtual reads | Plan Steps 8 and 15. Not this step. |
| C-2 | Coalescing holds across virtual repositories and across positions | Plan Steps 8 and 15. Not this step. |
| C-3 | The request's own state survives the shared instance | Plan Step 8. Not this step. |
| C-4 | Distinct paths still fill in parallel | Plan Step 8. Not this step. |
| S-1 | Upstream delete is refused | Plan Step 2. Not this step. |
| S-2 | Deleting the virtual repository removes its whole subtree | Plan Steps 1 to 3. Not this step. |
| S-3 | Namespace isolation | Plan Step 2. Not this step. |
| S-4 | Double association is impossible | Plan Step 2. Not this step. |
| S-5 | Reordering is transactional, and the deferral is what makes it work | Plan Step 2. Not this step. |
| S-6 | Range constraints bind | Plan Step 3 (the CHECKs). This step covers what happens when a value passes them anyway. |
Error cases
| # | Condition | Tests |
|---|---|---|
| E-1 | Manifest or blob write verb on a virtual repository: 405 | Plan Step 13. Not this step. |
| E-2 | Any verb on a blob-upload route: 405 | Plan Step 13. Not this step. |
| E-3 | No winner, an absence established at some position: 404 | Plan Step 11. Not this step. |
| E-4 | No winner, an absence established, another position ineligible for a data error: 200 or 503 | Plan Steps 10a and 10b. Not this step. |
| E-5 | No upstream left to consult: 404 | Plan Step 11. Not this step. |
| E-6 | Every upstream denied by well-formed rules: 404 | Plan Step 11. Not this step. |
| E-7 | Every rule-eligible position health-marked unhealthy: 503 | Plan Step 10b. Not this step. |
| E-8 | No absence established and no cache entry: 503 | Plan Step 11. Not this step. |
| E-9 | No absence established and a consulted position holds a cache entry: 200 | Plan Step 11. Not this step. |
| E-10 | Winner's Fetch answers a non-2xx after a successful Probe | Plan Step 11. Not this step. |
| E-11 | Winner's Fetch fails on transport before any byte is streamed | Plan Step 11. Not this step. |
| E-12 | Winner's Fetch fails on transport after bytes have reached the client | Plan Step 12. Not this step. |
| E-13 | Winner's body fails its digest check, headers not yet committed: 503 | Plan Step 12. Not this step. |
| E-14 | Winner's body fails the requested digest on a streamed blob GET |
Plan Step 12. Not this step. |
| E-15 | Content negotiation mismatch: 404 | Plan Step 15. Not this step. |
| E-16 | GET .../tags/list on a virtual repository: 404 |
Plan Step 13. Not this step. |
| E-17 | GET .../referrers/<digest> on a virtual repository: 404 |
Plan Step 13. Not this step. |
| E-18 | Abandonment mid-resolution | Plan Step 11. Not this step. |
| E-19 | Nothing eligible, at least one position ineligible for a data error: 500 | Plan Steps 10a and 10b. This step covers the per-upstream ineligibility that feeds it. |
| E-20 | A cache read or write the resolution depended on failed: 500 | Plan Step 11. Not this step. |
| E-21 | A coalesced follower outwaited its leader: 503 | Plan Step 11. Not this step. |
| E-22 | The upstream request was rejected before it was sent: 500 | Plan Step 11. Not this step. |
| E-23 | A tag-addressed manifest read whose tag fails the OCI tag grammar, or exceeds the 128 characters that grammar admits: 404 | This step. TestNewVirtualCoordinate_RejectsMalformedTarget (the grammar arm) and TestNewVirtualCoordinate_TagLengthCap (the length gate, at the cap and one past it). The matching spec row lands with !2126 (closed); until it merges this row describes the code without a spec line behind it. |
Security considerations
| # | Concern | Tests |
|---|---|---|
| SC-1 | No credentials of its own | Nothing in this step reads or holds a credential. Not tested here. |
| SC-2 | The ordering-control GA gate ([#792]) | A decision record, not behavior. No test. |
| SC-3 | Authorization is on the addressed repository | Plan Step 13. Not this step. |
| SC-4 | A composition can widen access, and that is the accepted model | Plan Steps 10b and 11. Not this step. |
| SC-5 | Position order decides which upstream can shadow another's tags | Plan Step 10b. Not this step. |
| SC-6 | The recursion guard is defended twice, on purpose | Plan Step 10b. Not this step. |
| SC-7 | Denied upstreams are indistinguishable in the response | Plan Step 11 (the response). This step pins the upstream-level denial the response hides: TestNewVirtualCoordinate_ImageRules, TestNewVirtualCoordinate_TagRules. |
| SC-8 | A digest is verified at every position that serves | Plan Step 12. Not this step. |
| SC-9 | No new outbound surface | Plan Step 9. This step issues no request. |
| SC-10 | Streaming bounds are the upstream's | Plan Steps 15 and 16. Not this step. |
Diff against the npm mirror
Per guardrail 6's mirror clause, against internal/format/npm/virtual_rules_test.go.
Row counts below are npm's, since the clause asks what the mirror asserts that
this suite might have dropped.
Carried over, predicate for predicate:
TestVirtualTargetFieldsandTestVirtualTargetFields_SharesNoBacking, both whole.TestNewVirtualCoordinate_PackageNameRules, all ten rows, asTestNewVirtualCoordinate_ImageRules(22 rows). npm's "admits its tarball too" and "matching an unscoped package" become the digest, blob, and single-segment rows.TestNewVirtualCoordinate_VersionRules, all thirteen rows, asTestNewVirtualCoordinate_TagRules(17 rows). npm's version and container's tag are the same shape: the field only one request shape addresses.TestNewVirtualCoordinate_VersionCaseSensitivity, all three rows, asTestNewVirtualCoordinate_TagCaseSensitivity. The OCI tag grammar admits uppercase and the image-name grammar does not, so the tag is container's one case-varying rule target. The matcher compares bytes, so a tag rule whose case differs neither matches nor excludes. The monolith reference matches case-insensitively underILIKE, and #911 tracks whether the two should agree. That issue carriesformat::npmtoday, and container tags are the first rule value whose case can differ.TestNewVirtualCoordinate_VersionLengthCap, asTestNewVirtualCoordinate_ImageNameLengthCap, on the field container can grow without a grammar bound. The tag needs none: the OCI tag grammar caps it at 128 bytes.TestNewVirtualCoordinate_CorruptRuleValuesFailClosed, all ten rows, widened to all three request shapes (14 rows).TestNewVirtualCoordinate_RejectsMalformedTarget, as the same-named test. npm asserts a per-casewantErrsentinel; container has one,ErrVirtualTarget, so the discrimination is not droppable.TestVirtualRulesFrom, all ten rows, plus one container-only row (11 total).
Deliberately not carried:
TestNewVirtualCoordinate_ScopeRulesas a group. npm's scope is derived from the package name and its legal value can be the empty string, so half that group asserts an addressed field carrying "". Container has no addressed field whose value is legally empty: the image name is never empty and no tag clears the tag grammar as "". The group's other half — a second defined field deciding a request on its own — is carried intoTestNewVirtualCoordinate_TagRules, and the empty-string half is replaced by empty-pattern rows on both fields.- npm's tarball-filename parse rows (
ErrTarballNameUnparseable, "widgets.tgz", "gadgets-1.0.0.tgz", "widgets-not-semver.tgz"). Container derives no rule value from a file name; the tag arrives whole. Replaced by the tag-grammar, kind-range, and tag/kind-consistency rows.
Added beyond the mirror:
TestNewVirtualCoordinate_FieldsAreNotTransposed, which gives the image and the tag disjoint values and matches each field's rule against the other's value. Every same-value fixture passes under a transposed key.TestNewVirtualCoordinate_AcceptsEveryShape, a control so the rejection table cannot pass by refusing everything.- Two rows in
TestNewVirtualCoordinate_ImageRulesfor a wildcard's depth. One*spans separators, soteam/*reachesteam/app/api. An image name has any number of segments where an npm scope has one, so the reach is wider here than the mirror had to assert.
Related to #291