fix(config): lower the storage.s3.chunk_size default to 5MiB

Stacked MRs

Part MR Scope Reviewable LOC Target
1/2 !3047 (merged) (merged) clamp the S3 upload buffer, one slot ratio for both drivers 451 main
2/2 this MR lower the storage.s3.chunk_size default to 5MiB 198 main

!3047 (merged) merged on 2026-10-12, so this MR is rebased onto main and targets it. This MR's 5MiB default derives a cap of 28 only with part 1's 1.2 ratio. On the old ratio of 2 it derives 18. LOC is added plus removed, less generated .pb.go files: part 1 at its squash commit 55fea6ee4, this MR at 1b0190c1c..39484bdc2.

Summary

At the 64 MiB default the derived upload-session cap on S3 is 2. So a third concurrent Maven deploy, npm publish, remote cache fill or single-POST blob upload waits 5s and gets a 503, and one npm packument rebuild takes both slots. A docker push joins that count once the cap plan's ResumeSession acquire (its step 12) lands, and one push has five layer uploads.

This MR lowers the default to 5MiB, the AWS per-part minimum. Under the 400 MiB fallback that derives 28, the same as the GCS default, and clears the 25 sessions of five concurrent pushes. 6MiB would derive 24.

On S3 it also refuses, at startup, a container.blob_max_size above 10,000 x chunk_size, the largest object one S3 multipart upload holds. The lower default brings that limit down to 52.4 GB, and without the check a larger cap loaded cleanly and a push failed only at part 10,001.

It also updates every place outside docs/plans/ and docs/adr/ that stated the old S3 figures: the configuration reference, docs/dev/storage.md, config.example.yaml, the run recipe, the S3Config.chunk_size proto comment, the caproni rig's comment on the cap it sets, the rendering-ceiling paragraph in internal/format/npm/packument_cache.md, and three npm test names and comments that called 2 "the S3 default cap".

Trade-offs of 5MiB parts:

  • Object size. S3 accepts at most 10,000 parts per upload, so 5MiB parts cap an object at 52.4 GB. That clears the 50GB container.blob_max_size default (9,537 parts). Startup refuses a container.blob_max_size above 10,000 x chunk_size on S3 and names both values, so an install that raised it past 52.4 GB fails at boot, not at part 10,001 of a push.
  • Request count. About 13x more part uploads per byte. The S3 request limiter (max_requests_per_second, 350 by default) then caps one process's part uploads near 1.7 GiB/s.
  • Resumes. A resumed upload whose last part is short, like a multi-PATCH OCI upload, re-copies its object on the server in chunk_size parts. At 5MiB that is about 13x more UploadPartCopy calls per resume. A client that sends a layer in several PATCH requests pays it on each one.
  • Shared request limiter. The re-copy runs up to multipart_copy.max_concurrency (100) UploadPartCopy calls at once, and each waits on the one S3 request limiter the process holds (max_requests_per_second, 350 by default). So a re-copy delays every other S3 call on that pod, for every tenant. Example: a 10 GB blob sent in ten 1 GB PATCH requests re-copies 45 GB in total, about 675 calls and 2 s of limiter time at 64MiB, and about 8,600 calls and 25 s at 5MiB. The cost grows with the square of the PATCH count. The hazard exists on main, and this default makes it about 13x cheaper to trigger. It is accepted here, and #1589 (closed) tracks re-copying in multipart_copy.chunk_size parts instead.
  • Explicit settings stay. An install that sets chunk_size keeps its value. config.example.yaml used to set "64MiB" explicitly and now sets "5MiB".

Governing ADRs

scripts/adr-freshness.sh: the mirror is current.

  • ADR 003 (docs/adr/003_system_requirements.md): its storage-throughput paragraph says the S3 driver at its shipped defaults "admits too few sessions to reach the projection". With this MR the S3 default admits what the GCS default does, so it conforms, and that sentence goes stale (see below).
  • ADR 004 (docs/adr/004_data_and_application_limits.md): container blobs up to 50 GB. 5MiB parts hold 52.4 GB, so the ADR's limit still fits. Maven and npm are 5 GB.

Testing

  • Config tests: the S3 default loads as 5242880 bytes and derives 28. Setting the default back to 64 MiB reddens both.
  • The S3 part-limit check: one byte past 10,000 parts of the 5MiB default is refused, the exact figure loads, a 64MiB chunk_size holds 600GB, and GCS is not bound by it. A unit test pins that 10,000 parts of the S3 default hold the 50GB blob default. Removing the check, changing > to >=, dropping the S3-only guard, or raising the blob default to 60GB each turns its test red.
  • The renamed npm tests pass (TestSetRebuildRenderingCap_ClampsTheCeilingDownNeverUp, TestRebuildStreamedKinds_ReturnsBothUnitsWhenTheWindowFails, TestReserveRenderingSlot_ShedRecordsTheCeilingInForce).
  • Unit tests pass for cmd/artifact-registry, internal/config, internal/format/npm, internal/storage/..., internal/accounting and internal/managementapi, the packages that load config.example.yaml or read the cap.
  • driver.sh smoke at dd06f688a: 103 passed, 0 failed. Its boot line reads driver s3, chunk_size_bytes 5242880, max_concurrent_upload_sessions 28, floor_bound false. The startup check landed after that run. The recipe sets no blob_max_size, so it boots on the 50GB default, which 10,000 5MiB parts hold.
  • golangci-lint 2.14 reports 0 issues on internal/config and internal/format/npm, and the full pre-commit chain, including the comment-caps gate and buf generate freshness, passes on every file the branch changes.

The run recipe changes in this MR because guardrail 20 asks for it in the same MR as the boot change. It is a one-sentence fact correction, so it carries no skill benchmark.

No e2e scenario is added or affected: e2e.maven.publish.upload-slots-exhausted sets its own cap of 2, and no scenario relies on the S3 default. No docs/user/ page covers storage.* settings, so no user documentation is affected.

Not changed here

  • npm.max_tarball_size has no upper bound and is not checked against the S3 part limit: the startup check covers container.blob_max_size only. Maven's max_artifact_size is capped at 5 GB, which any S3 chunk_size holds.
  • ADR 003's S3 shortfall paragraph needs a handbook ADR update once this stack lands.
  • docs/plans/2026-09-03-upload-session-concurrency-cap.md still records the 64 MiB default and cap of 2 as its basis, including the Step 12 arithmetic part 1 repriced for the 1.2 ratio. Left alone: this MR is not a step of that plan.

Closes #1171 (closed)

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports

Loading
Loading