feat(managementapi): meter the listing filter's outcomes and refusals (S09 LookupResources plan: 4/4)

Instruments the repository-listing authorization filter Step 3 built. The denied branch had no series of its own: a grant drain refusing every list looked identical to a healthy one, because authz_denials_total books nothing for any of its arms by design.

Three metrics, all under gitlab_artifact_registry_. The surface stays out of the names: the filter is surface-independent, and a second listing surface (an OCI catalog endpoint, if one ships) adds a listing_surface label then, which existing queries survive where a rename would not.

  • authz_listing_filter_failures_total{listing_failure_reason} — the alert surface. One per drain refusal, keyed to the arm that refused: grant_ceiling, page_ceiling, budget, transport, identity. The client sees the same envelope for all five, so the reason is what separates a ceiling that needs raising from an IAM outage that needs paging. budget and transport split the same LookupResources error on the drain context's own cause. Booked where the drain is refused, ahead of the disconnect check, so a caller that hung up mid-drain still counts (as transport — an inherited cancellation is not the budget); the 503 and its ERROR stay suppressed for a gone client, and the budget/transport arms carry their own namespace-bearing WARN as the backstop.
  • authz_listing_filter_outcomes_total{listing_outcome} — the behaviour shares: unrestricted (the ancestor hatch lifted the restriction), filtered (a page served under the drained grant ids), empty (no usable grant). unrestricted climbing means the escape hatch is absorbing the traffic the restriction was meant to bound.
  • authz_listing_drain_duration_seconds — label-free, one observation per drain, success and failure alike. It is the headroom reading against the drain's own 20s budget: a p99 climbing towards it is the warning ahead of the budget refusals.

The outcomes counter books one per listing the branch settled, which is not quite a count of 200s: a rendering failure after the branch returns (the verdict batch, the remote-settings read, serialization, the pagination links) answers 5xx and still books one. Those are defect and outage paths, and the catalog row says so rather than claiming a denominator the code does not produce.

authz_denials_total is untouched; its row now points at the failures counter as the series that does move, and the existing test pinning its silence still passes.

Both label domains are pinned closed in internal/metrics/cardinality.go and seeded from this package's own constants in a new audit test. The two packages do not import each other, so pin and constant are independent strings that the audit only compares where a gather carries the value — misspelling either side ships green without the seed.

Every arm has a capture-before/assert-after case on its own series plus an assertion that the sibling vector did not move, and two absence claims are pinned the same way: a chunk-path 503 books no reason, and a disconnected drain refusal books the counter while the response stays suppressed. Ten injection proofs back the suite: dropping either increment, dropping the observation, observing a constant instead of the drain's wall clock, collapsing the empty filtered page onto empty, misspelling one seeded label, tagging each of the three chunk-path wraps with a drain reason, and moving the booking back below the disconnect check each redden it.

Diff is 655 added lines against the 500 guardrail, and splitting would not help: 169 are Go source across three files, 454 are the metric suite and the audit seed, and 32 are the catalog and plan rows. The suite is the part over the line, and it is per-arm by construction.

No e2e scenario is added or affected: nothing about the served response changes, and docs/testing/e2e/management.md has no repository-listing visibility scenario.

Depends on !2452 (merged) and targets its branch: it instruments the arms that MR builds. Retargets to main when !2452 (merged) merges.

Closes #1281 (closed)

Related to https://gitlab.com/gitlab-org/ops/artifact-registry/-/work_items/969

Edited by João Pereira

Merge request reports

Loading
Loading