feat(accounting): add the lease-held gauge and pin the emit result set

What this delivers

A label-free per-pod gauge now says whether this process holds the storage-accounting metrics lease, and counter_emits_total's result label is pinned to the six values its budget enumerates. The documentation for both changes moves with them, together with the issue's other doc/code corrections.

Cause.

  1. No series said which pod holds the storage-accounting lease. wireAccountingMetrics built the lease and registered only the two collectors the lease gates. Both emit nothing from a non-holder. So a fleet with no lease holder looked the same as a fleet whose lease-gated gauges were only absent.
  2. counter_emits_total's result label was budgeted but not pinned. internal/metrics/cardinality.go carried "result": 6 in expectedDistinctValues and no result entry in closedSetValues. A value outside the six-value union therefore passed the audit for as long as the distinct count stayed inside the budget.

What the change does.

  • internal/accounting/metrics.go: LeaseHeldCollector.Collect emits one label-free gauge sample per scrape, read from the lease's Held(). It emits whether or not this process holds the lease. Its doc comment counts the three gauges the lease gates.
  • cmd/artifact-registry/wire_accounting_metrics.go: wireAccountingMetrics registers accounting.NewLeaseHeldCollector(lease) beside the two gated collectors, over the same lease.
  • internal/metrics/cardinality.go: result joins closedSetValues, pinned to {ok, error, panic, dropped, superseded, timeout}. The three "inert" comments now say that they describe the series the cardinality audit gathers, not production.
  • docs/dev/storage-accounting.md: jobs_processed_total and jobs_errors_total list backend and queue. The drain path's three Warn sites are recorded as kept log-only, with the reason. The collector census counts both non-pipeline collectors and describes accounting_lease_held. The paging-alerts known gap now says the rules are specified but not written or deployed, and points at #1662, not at #354 (closed).
  • docs/dev/alerting.md: the intro names the npm and the OCI downloads_count rules. The rules' home file exists, so the work is an addition to it. The decision not to add a rollout-window series is recorded, and #1600 (closed) is cited as where it was taken. The claim that no series reach the Mimir tenant is replaced.
  • docs/dev/observability.md: a catalog row for gitlab_artifact_registry_accounting_lease_held, the drain drops recorded as log-only, and the metric that records a failed count.

Coverage

docs/specs/S22-storage-accounting.md (the card's spec) is Status: Implemented and carries no Acceptance Criteria, Error Cases or Security Considerations; it points at docs/dev/storage-accounting.md. The rows are therefore the issue card's acceptance items, in card order.

# Criterion (issue card acceptance) Tests
A-1 storage-accounting.md:1432,1434 list backend and queue on jobs_processed_total and jobs_errors_total. Doc-only edit; no test.
A-2 The alerting.md intro names both the npm and the OCI downloads_count rules, or points at ### Required alerts. Doc-only edit; no test.
A-3 result is pinned in closedSetValues, or the reason it stays budget-only is recorded in cardinality.go. TestClosedSetValues_PinTheCounterEmitResultVocabulary (internal/metrics/cardinality_test.go).
A-4 The "inert" comments in cardinality.go and internal/accounting/metrics.go say they describe the audit, not production. Comment-only edit; no test.
A-5 #435's description names reconciliation_drift_rows. An edit to another issue, not a repository change (card, Resolver findings); no test.
A-6 A series for the counter_drain_chunk_downloads rollout-window loss exists, or alerting.md records the decision not to add one. Doc-only edit (operator decision F4: no series); no test.
A-7 For each log-only signal above, a metric exists or the doc records the decision to keep it log-only. Lease-held gauge: TestNewLeaseHeldCollector_PanicsOnANilLease, TestLeaseHeldCollector_ZeroValueHasNoLeaseToRead, TestLeaseHeldCollector_ExposesOneLabelFreeGauge, TestLeaseHeldGauge_FollowsTheLeasesHeld, TestWireAccountingMetrics_RegistersTheLeaseHeldGauge. Drain drops (F5) and failed counts (F6): doc-only edits; no test.
A-8 alerting.md:445-450 names #1662 instead of #354 (closed) and drops "no series reach the Mimir tenant"; :257-263 no longer says the rule file has to be created. Doc-only edit; no test.

Error cases: none — the card's list carries no error case and the change adds no failure path. Security considerations: none — the gauge is label-free, so it exposes no namespace, repository or pod identifier.

E2E scenario catalog

This merge request adds no row to docs/testing/. The change adds a scraped gauge and documentation edits, and it changes no user journey, endpoint, route or protocol. No scenario is added or affected.

Plan deviation

No plan governs this work: no plan under docs/plans/ declares S22. This feat issue change therefore states that deviation from the plan-before-code rule here.

Merge order

  • !3452 conflicts with this branch in docs/dev/alerting.md and docs/dev/observability.md. It merges cleanly into origin/main. Whichever of the two merges second rebases.
  • !3451 conflicts with this branch in docs/dev/observability.md. It merges cleanly into origin/main. Whichever of the two merges second rebases.
  • !3380 (5/5 of its stack) conflicts with this branch in docs/dev/observability.md. It merges cleanly into origin/main, so this conflict is not its own staleness. Whichever of the two merges second rebases. !3401 and !3402 from the same stack merge cleanly with this branch.
  • !3249 (S05-B rate limiting) and this branch each append one test at the end of internal/metrics/cardinality_test.go. This branch pins result. !3249 pins limiter. Whoever merges second keeps both tests.
  • !2944, !2971 and !3226 conflict with origin/main on their own, so those conflicts are their own staleness. The test_datastore failures in the !3380 and !3402 unions are their own query_names.go constants, and this branch touches no internal/datastore path.
  • This branch also conflicts with the current origin/main in docs/dev/observability.md, so it needs a rebase before it merges.
  • Other open merge requests edit the same docs/dev/ pages: !2944, !2971, !3226, !3249, !3348, !3380, !3401–!3403, !3451 and !3452. Whichever merges second re-checks that wording.

Pre-existing on the base, and not a merge-order item: the lint-integration findings the branch battery reports sit in cmd/artifact-registry integration test files that are byte-identical to origin/main at fb0dd4d90263d841af0ab2be131204932fd5ff18. The repository's integration lint is delta-scoped, so CI is not affected.

Closes #1600 (closed)

This is a bot message 🤖 — /smurfit

Edited by Pawel Rozlach

Merge request reports

Loading
Loading