Harden the usage-data emission path for request-rate traffic

Implementation plan (authoritative for design decisions and the full guard census): docs/plans/2026-08-03-usage-data-emission-hardening.md, landing via !1260 (merged). The corrections below defer to it.

Problem

S23 moved usage-data emission from admin-rate call sites (repository create and delete) onto request-rate ones (Maven, npm, and Container/OCI push, pull, and delete). Three properties of the shared emission path were sized for the admin-rate case and stop holding at request rate. All three surfaced in review on the Maven and Container/OCI steps and were deferred so the format steps could land.

What to fix

1. The anchor-gate skip log is unbounded

trackArtifactEvent logs at ERROR whenever a namespace has no parseable organization anchor, or its repository kind has no usage-data mapping. Every one of those conditions is a property of the namespace row, so a single misconfigured namespace produces one ERROR line per request for as long as it stays misconfigured. On a pull path that is CI rate, and it can bury real errors.

The same guard also logs when usage_data.enabled is false: usagedata.New returns a non-nil no-op tracker, and the handler guard tests tracker == nil, so the gate runs and logs even though Track does nothing.

Deliver: bound the log, carry the durable signal on a counter instead, and stay silent when the tracker is inert. (The plan bounds per (source, event, reason) rather than per namespace; rationale there.)

Open decisions this work owns (settled in the plan):

  • Whether the counter lives on usagedata.Tracker next to dropped (shared with every consumer, but widens each consumer UsageTracker interface and its wire assertions) or stays per package.
  • Its label set. A namespace label needs the cardinality review docs/dev/observability.md requires.
  • Whether ERROR drops to WARN once a metric carries the signal.

Four call sites carry the same copied guard and must move together: internal/managementapi, internal/format/maven, internal/format/npm, internal/format/oci.

2. The nil-tracker boot guard cannot fire

w.usageData is a *usagedata.Tracker. A nil pointer of that type boxed into a consumer's UsageTracker interface yields a non-nil interface, so both the wire* boot guards and the handlers' tracker == nil checks pass a typed nil straight through. (*Tracker).Track then nil-derefs after the response is already written.

Not reachable today, because wireUsageData runs first in wireServices and its error aborts boot. It becomes reachable the moment the composition root is reordered.

Deliver: concrete-typed wire seams (production builders take *usagedata.Tracker, so their existing nil rejections become effective), plus a nil-receiver arm in the tracker itself. The census is larger than first written here: five ineffective interface-level guards plus the boxing seam in wire_root_dispatcher.go; wire_management.go:33 already checks the concrete type, and UsageTrackerWired guards a different invariant (a dropped handler option) and stays. A bare wireServices nil assert was evaluated and rejected as unreachable and untestable; see the plan.

3. pending never self-corrects after ring-buffer overflow

Track increments pending; confirmDeliveries decrements only on confirmed successes. The emitter's ring buffer overwrites the oldest event when full without firing a callback, so after one overflow pending overstates the queue permanently. awaitDelivery then loops until its context expires, so Shutdown (which runs once per process) burns its full drain budget and returns an "events undelivered at shutdown" error even when the queue is empty: one 5-second stall and a spurious error per pod termination.

Request-rate emission also shares the buffer with repository_created and repository_deleted, so a pull storm can now evict admin events that never competed for slots before. That changes the loss profile of already-shipped signals.

Deliver: reconcile pending against gitlab_artifact_registry_snowplow_events_storage_overflow_dropped_total so the drain budget recovers instead of staying poisoned.

Rollout gate

Before usage_data.enabled goes true against production traffic:

  • Size the ring buffer against measured per-pod push and pull rate, against the emitter's drain ceiling. Blocked on upstream LabKit configurability: ring and batch sizes are hard constants in v2.29.2; the plan files the upstream ask.
  • Alert on ..._storage_overflow_dropped_total, and on ..._storage_queue_depth approaching ..._storage_queue_max_size. All three are already catalogued.

Out of scope

  • The auth_method dimension, which waits on the auth context reaching handlers.
  • Any change to the ADR-012 event catalog or its dimensions.
Edited by João Pereira