test(e2e): the five instance-configuration rows, and three defects a rig found

Five new rows for the instance-configuration profile, all run green on a real rig: junit-instance.xml reports 31 tests, 0 failures, 116s for the whole profile. Coverage 158 → 159, 0 disagreements, nothing built-but-unrun.

boot.refuses-metadata-over-artifact-cap       PASS 12.4s
boot.refuses-manifest-over-blob-cap           PASS 14.9s
boot.refuses-unenforced-with-iam              PASS 12.9s
shutdown.drain-completes-inflight             PASS 11.7s
shutdown.restart-preserves-cached-content     PASS 42.3s

The boot rows needed an arm the driver did not have

A config that fails validation means the process exits 1 and the pod never becomes ready — so apply → rollout status → behavioural gate cannot work, because the rolling update holds the old pods while the new ReplicaSet crash-loops. WaitUntilServing therefore succeeds under a refused config, and a behavioural check would pass against the very configuration the variant was meant to replace. The ordinary gates are not slow here, they are wrong.

The arm detects the refusal in 6–9s off the pod phase rather than waiting out a 10-minute rollout timeout, and matches the validator's own text from the service's structured boot record.

That reason-matching guard earned its keep twice on the rig, and both failures were a refusal for the wrong reason rather than a missing non-convergence:

• iam: {address: …} → iam.service_token: value is required, because protovalidate runs before the pairing check.
• service_token: "<string>" → expected fields for SecretRef, got scalar, because it is a SecretRef with a required oneof.

Only service_token: {env: …} reaches the pairing check at all. An arm asserting the rollout merely failed would have reported that row green on both wrong configurations.

The drain row takes a pod-scoped forward, and one replica would not do

Readiness is on the probe listener (9090); the Service publishes 8080 alone. More fundamentally, a draining pod leaves the Service's endpoints the moment readiness fails — that is the gate's purpose — so a Service-scoped forward stops reaching the pod at the exact instant the row observes. No replica count changes that, and with no strategy set a rollout surges by one regardless. Cluster.ForwardPod addresses pod/<name>; the upload is addressed at the same pod, since one spread across replicas would not be in flight on the instance being drained.

Both reasons are recorded in e2e/README.md, the row's cell and the method's doc, because "just use one replica" is what a reader will reach for.

The 5-second window has two distinct ways to be missed, and both are separate sentinels pinned against a stub in 15ms: ErrNeverUnready ("stayed ready for 15s") and ErrPodGone ("stopped answering before readiness went unready"), with a third arm for never answering. Injection-proven, so the messages provably differ rather than being believed to. The row also asserts the upload is still in flight when the pod is stopped — one already committed would satisfy the surviving-upload clause without the drain being crossed.

Two probe defects the rig found in the already-merged driver

• The body-cap probe could never see a 413. It sent repeated "a" under application/json, so the decoder rejected the first byte and answered 400 at every size, 6 MB included. MaxBytesReader trips as the body is read, so the body must be well-formed. The surface would have hard-failed on every run. Boundary now measured exactly.
• Its sizes were literals that the body-caps change invalidates, putting the probe's value exactly on the cap. They now derive from the variant's own cap, with a test reading the rig's declared cap out of its values file.

The bodycap surface is removed: !2486 (merged) lifts those rows by lowering the rig's own cap, which is the cheaper answer. The variant stays for a row no single rig value can serve.

deployment joins the Interface vocabulary: none of the twelve existing values describes what a boot row drives, since a refused process serves no request, and HTTP/API would both be mislabels.

Deletes the drain row's own limitations.md entry, per that file's rule.

Four more defects, found by review rather than by the rig

Duo and the AppSec review landed on the harness rather than the rows. Three of the four Duo findings were real:

• refusedPod matched the exit code by prefix. strings.Contains(state, "\t1") also matches 10, 100 and 137, so an OOM kill during startup read as the configuration refusal a boot row asserts — and the rig values file records this container OOM-killing at both 512Mi and 1Gi, which is why its limit is 2Gi. The comparison moved into refusedIn, a pure helper over the read, so the codes it has to reject are pinned by a test rather than argued (e16d80bf).
• The drain window's "answered then stopped" subtest never took its own branch. It closed the probe server before awaitUnready ever probed, so the handler never ran; both branches return ErrPodGone, so nothing in the subtest could tell them apart. It now closes from inside the handler and asserts the poll count only the answered branch reports (c8484888, aa3f2770).
• The e2e/README.md tree omitted the new surfaces and left drain.go out of the profile's file list (84d8c7ab).

The fourth was invalid: scenarioid is regenerated in the diff, and the constant is spelled ...WithIam. Established by compiling rather than reading, since these files sit behind //go:build e2e — go vet -tags=e2e ./e2e/... and go test -tags=e2e -run '^$' ./e2e/... both clean.

One further defect came out of the AppSec review, in this stack's own new code:

• Forward.await tested reachability, not ownership. It waited only for a local port to accept a connection, and with the drain's ports fixed at 18080/19090 a forward surviving an earlier run satisfied that wait while this call's kubectl port-forward failed to bind — so a row addressed the earlier run's pod while reading green. It failed toward a wrong answer rather than an error. The ports are now bind-checked before kubectl starts, under a sentinel of their own, because a held port and a forward that never carried traffic want different answers (6677c858).

Both AppSec findings themselves were declined on the merits, in the confidential thread.

One drive-by, in a test this branch does not otherwise touch:

• TestDeployDropsTheKubectlSubcommand raced its own stub. It writes an executable stub and immediately execs it, and while it ran in parallel with the sibling tests that shell out through sh -c, one of their forks could inherit the still-open write descriptor — the exec then failed ETXTBSY. It is not theoretical: it failed the arm64 image build, whose Nix sandbox runs the suite, and took the merge train down with it. Measured at 1 failure in 30 package runs before and 0 in 60 after running it sequentially, with the mechanism accounting for the numbers rather than merely correlating. Taken here rather than split out because it blocks this branch's train and lives in a file this branch already modifies.

Justification for 1,515 insertions (guardrail 17, measured at a818ee54f): go non-test 878, tests 509, docs 126, config 2. Splitting would separate the boot arm from the only rows that exercise it.

Related to #1289

---

The implementation plan for this initiative is maintained out of tree by decision, so the (instance-profile plan: N/4) markers in this branch's history name a plan that is not in the repository.

🤖 Generated with Claude Code