Amend AR ADR-003: the upload-session cap bounds the projection's upload half

Dzmitry (Dima) Meshcharakourequested to merge
adr-003-upload-session-concurrency-cap into main

What this amends

ADR-003's GitLab.com projection line, "Storage throughput: hundreds of MB/s upload, several GB/s download", keeps its one-line form. A paragraph after the projection lists qualifies it: the figure is service-wide rather than a per-session rate, its upload half is bounded by the concurrent upload sessions a pod admits, and at least one supported storage configuration does not reach it.

Why

The Artifact Registry is adding a global cap on concurrent upload sessions, to keep the staging buffers those sessions hold inside a pod's memory budget. The cap is derived from the storage driver's chunk size:

  • GCS at its 5 MiB default reserves 7 MiB per slot, so the cap is 28 sessions per pod.
  • S3 at its 64 MiB default reserves 129 MiB per slot against a 200 MiB budget, so the floor binds and the cap is 2 sessions per pod.

At the two-replica floor a default S3 install therefore admits 4 concurrent sessions service-wide. Reading the projection as 200 to 1000 MB/s upload, that session count asks 50 to 250 MB/s per session on the upload half. That range is not met on that configuration, so the projection is unmet there rather than merely tight. The same install also over-admits the budget the cap was derived from: two slots at 129 MiB is 258 MiB against 200 MiB, so there the cap does not keep the staging buffers inside the budget. The amendment records both misses.

The cap bounds the upload half and not the download half. A download opens no upload session and holds no staging buffer, so the admitted session count is not a bound on downloads; the plan applies the same per-session arithmetic to the download projection as a plausibility test, and the ADR says the cap is not a bound there rather than reporting a download shortfall.

The behaviour itself stays within ADR-003's responsiveness goal: a request that finds no free slot waits a bounded time and then receives a 503 the client can retry, which is predictable degradation rather than widespread failure. What this MR changes is the projection's text, so the deviation is recorded where the projection is stated instead of only in the implementation plan.

The configurations that do reach the projection, the GCS default among them, are recorded as passing on a judgement rather than a measurement: the top of the per-session range they imply is a rate a single transfer sustains, which nobody here has measured. The amendment says so, and says that a single-session reading below that top reopens the question, so the two verdicts are not read as equally evidenced.

What stays out of the ADR

The chunk_size arithmetic that derives the cap, the per-slot reservations, and the boot warning the shortfall raises are not in the ADR. They live in the Artifact Registry's configuration reference, under storage.max_concurrent_upload_sessions and storage.s3.chunk_size, which is where an operator reads them.

Both questions this description used to put to reviewers were answered in review: qualifying the projection with an implementation-imposed bound belongs here, the knob arithmetic does not, and recording the passing configurations as a judgement is the right standard at this depth.

Merge order

This amendment is a merge gate on the Artifact Registry side. The steps that turn the cap on must not merge before it:

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports