HAProxyCfg publication recreates every unchanged auxiliary file under a new set name
Every publication of a new auxiliary set recreates all auxiliary children, including the ones whose content did not change, and then deletes the previous set one child at a time. At 800 Ingresses that is most of the remaining HAProxyCfg publication latency.
Measured
TestScale on an isolated kind cluster: 800 Ingresses, 20 Gateways, HAProxy 3.4. One HAProxyCfg set holds 74 children: 61 map files, 10 general files, 1 crt-list and 2 Secrets.
A one-Ingress change touches one or two maps, but every child name carries the set ID (auxiliaryResourceSuffix(setID) in pkg/k8s/configpublisher/name_helpers.go). Each change therefore costs, from apiserver request metrics over 20 samples with !1959 (closed) applied:
| per change | count | apiserver latency each |
|---|---|---|
| child create (GET 404 + POST) | 74 | ~20 ms POST |
| stale child delete, serial, each behind the publication fence | 74 | ~13 ms DELETE + ~8 ms metadata GET |
| per-pod status stamp (SSA) on every new child | 2 x 74 | ~20 ms |
After !1959 (closed) the creates and stamps run 8 at a time. The deletions stay serial: a parallel prune would widen the window in which a superseded publication can still delete children. They take about 74 x 21 ms ≈ 1.5 s per change. A back-to-back change waits behind the previous publication's prune, so publication sits at about 2.1 s median (0.65 s from idle), while routing converges in about 0.2 s.
The stamps also can't be elided: the aux stamp cache (aux_stamp_cache.go) is keyed by child name, and every name is new.
Direction
Name children by content instead of by set, for example <base>-<hash(path, content)>. An unchanged file then keeps its object across sets: no create, no delete, and its pod stamps stay elided. Only changed files would churn.
What reader set verification would need to change
Readers (pkg/controller/publishedauxfiles.go) resolve the committed status.auxiliaryFiles and require the haproxy-haptic.org/auxiliary-set-id annotation on every referenced child to equal the commit's set ID (validatePublishedSetID). A shared child can't carry one set ID. Verification would need to move to per-child content identity:
- Each child carries its content checksum (already annotated as
haproxy-haptic.org/checksumon Secrets). Readers check it against the hash embedded in the referenced name, which makes a child immutable by construction. - The commit stays the status update that names all references together, so set completeness still comes from the commit, not from the children.
- Cleanup can't delete a child that is still referenced by the current commit. With shared names the prune's "not desired" set already excludes it. The A→B→A race needs re-checking because a re-published set reuses existing objects. Today's fence narrows that race but doesn't close it.
- Migration: existing set-suffixed children must still resolve until the first content-named publication commits.
This is a protocol change to a validation path (RULE #2), so it needs an ADR and an argument that the new verification is at least as strong as the set-ID check.