docs(plans): soak the four whole-transfer upload paths, not docker

Summary

This amends the Step 11 staging soak in docs/plans/2026-09-03-upload-session-concurrency-cap.md, and moves two items from Step 11 to Step 12. It is the plan side of this review thread on !2882.

Why: at Step 11 a chunked docker push holds a slot only for its first POST. Its PATCH and PUT hold no slot until Step 12. So the soak's docker load leaves the slot gauge near zero, run B cannot shed, and the memory bar reads staging the gate does not bound.

What changes:

  • The soak load. It is now four paths that hold a slot for the whole transfer: a Maven deploy, an npm publish, a remote fill, and the OCI single POST. Clients split evenly across the four, 7 each on run A and 14 each on run B. Each client keeps one request in flight, and each request carries one unique payload of at least 6.5 MiB.
  • The bars, re-derived for that load.
    • Run sizes stay at 28 and 56. They come from the cap, not the client.
    • The memory bar stays at 336 MiB, and the plan says why per term. The 111 MiB lazily-returned term was measured under a docker push, so run A is its first reading under this mix.
    • The shed bars read the four client sources only. npm_rebuild is recorded but not barred: every publish dispatches a rebuild that asks a full gate for two units and answers no client.
    • The 60 s hold and the 4-minute turnover window stay. All four paths advertise the same 28 to 42 s Retry-After, so the ~40 s return cycle they rest on still holds.
  • The docker push arithmetic moves to Step 12. That is the configuration-reference sentences and the floor-binding warning's availability half. Step 11 now records what part 1 writes instead.
  • The concurrent-push e2e scenario moves to Step 12. Step 11 now records the Maven upload-slots-exhausted scenario that part 1 adds.

From review:

  • Run A's bar is turnover, not zero sheds. A two-unit npm rebuild at the head of the queue can make a client shed on a healthy gate, so run A now asks every client to get in once in any 2-minute window, the same two-turns derivation as run B's 4 minutes.
  • The soak's conditions are pinned further: the generator drives one pod's address, runs happen in a quiet window, and full-speed readings of each hosted upload and remote-fill format are read against the plan’s 54 and 18 MB/s thresholds.
  • Step 12 needs no soak of its own, and the plan states when it would.
  • Step 11 gains a gomemlimit_fallback boot field and a note that the cap also covers read paths, on the configuration row and in config.example.yaml.
  • ## Approach records why the gate counts sessions, not bytes.
  • Step 11 wires the npm rebuild's gate and cap at the composition root. wireStorage returns the gate it builds, and wireServices hands it and the already-loaded cap to the two rebuild setters. That is one widened seam at 12 call sites instead of four seams at 37, and it keeps npm wiring out of wireStorage.
  • Factual slips are fixed: two S3 slots at a 5 GiB chunk_size reserve about 20 GiB, not 10; the middleware reason now matches the research; and two cross-references are corrected.

Every site that carried these facts is fixed: Step 1's floor-warning paragraph and its Shared seams; Step 11's configuration-reference, storage.go and e2e bullets and the soak bullet; Step 12's Scope, Files, Shared seams and Forecast; and the e2e paragraph under ## Testing Strategy.

What this does not reach

  • No soak result. The soak is not run here. It runs against !2882 (merged)'s branch under the load this MR pins, once review settles that load, and the run parameters, verdicts and series (or links) go in !2882 (merged)’s description.
  • No soak for a chunked docker push. That population starts holding slots at Step 12, and the plan now says Step 12 needs no soak of its own unless it changes the reservation, the fraction, or what a resumed session stages.
  • Other part 1 text that Step 12 makes false. Part 1 also says in docs/dev/storage.md and docs/dev/observability.md that the resume path takes no slot. This MR names only the configuration row. Step 12 rewrites the rest when it lands the resume acquire.
  • ## Dependencies and the research sections still state the S3 availability half in docker push terms. They describe the plan's end state, after Step 12, so they are left as is.

Merge order

This MR is a merge gate on !2882 (merged), set as a merge request dependency. Merge this one first.

One other open MR edits this file: !2885 (merged). git merge-tree --write-tree of the two heads (359848297 and 90ce0fe2d) is clean, so either can land first.

Governing ADRs

  • ADR-003 (system requirements): the plan's deviation from the throughput projection is recorded by handbook!21192, which has merged. This MR moves no term of it. The cap, the 0.5 fraction and the reservations are unchanged; only the load the soak uses to check the fraction changes.
  • ADR-004 (limits): unchanged. The soak is per pod and says nothing new about per-tenant limits.
  • ./scripts/adr-freshness.sh reports the mirror one upstream commit behind, on ADR-020 (authentication flow). This MR does not touch auth.

Testing

Docs only, one file: 869 reviewable LOC (595 added, 274 removed) at 359848297 against merge base f7f158cd9. The review follow-up adds 258 reviewable LOC (181 added, 77 removed) in that plan file. Splitting would separate corrections to the same Step 11 evidence and remediation contract from the load they govern; each follow-up answers review of this amendment. The full pre-commit and commit-message hook chains pass, including markdownlint, vale, lychee, gitleaks and gitlint.

Latest review corrections

  • Named the soak owner, !2882 (merged) description as its verdict record, required evidence, and the blocked outcome when calibration cannot clear the bars.
  • Distinguished rebuild contention from budget shortage before a fraction raise; an inconclusive diagnosis does not authorize a raise.
  • Corrected CPU scaling claims and checked the 2–20-pod HPA range. S3’s combined projection still fails the plan’s test; its upload half alone is no longer ruled out at 20 pods.
  • Described Retry-After as an optimistic convention, not a slot-availability estimate; made Maven retry behavior conditional on the driven client.
  • Reconciled the research bases, configured-cap INFO record, source parameter, acceptance count, shared seams and realized per-part forecasts.
  • Compared enforced S3 admission with the uncapped baseline and recorded the ordinary-client shed escalation trigger on #1171 (closed).
  • Added per-format full-speed readings and stated that the controlled soak does not establish a routine-production concurrency baseline.

ADR freshness was checked again: the four unsynced upstream commits affect ADR-007/008/009/020, not governing ADR-003/004. The sibling !2885 (merged) still owns the handbook amendment’s upload-only scope correction; git merge-tree between its head and 359848297 is clean.

The code claims in the new text were checked at !2882 (merged)'s head 5a078d5b6:

  • internal/format/oci/upload.go:865, where handleInitiate defers the release, and :1979, the resume arm, where ResumeSession takes no unit.
  • The shed Retry-After on all four paths: a 35 s base with a jitter of a fifth, so 28 to 42 s. See internal/format/oci/upload.go, internal/format/maven/upload.go (also drawn by the Maven remote fill in internal/format/maven/remote_errors.go) and internal/format/npm/publish.go.
  • The admission_source values in internal/storage/metrics.go.
  • The max_concurrent_fills_per_repo default of 16 in config.example.yaml.

No e2e scenario is added or changed by this MR. It changes the plan only.

Related to #1035

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports

Loading
Loading