chore(metrics): create HTTP server series at startup
What this delivers
This MR is part 1 of 4 of the split of !3348 (merged).
A new instance now exports the HTTP server series at 0 before it gets any traffic, so the HTTP dashboard panels read 0 and not "No data".
Cause.
A Prometheus *Vec exports no series until a child exists.
Before this change, only HTTPServerMiddleware created the HTTP children, so a fresh instance had no HTTP series until traffic arrived.
Fix.
metrics.InitHTTPRoute creates the children of one route with WithLabelValues and records no observation.
A histogram child exports _count 0, and no code calls Observe(0).
metrics.InitHTTPPattern seeds a ServeMux pattern at status_code 200 and 500, with the method from the pattern, else GET.
The seed calls sit beside each Handle call whose pattern reaches the route label:
| Pattern | Where the call is |
|---|---|
| The OCI token endpoint | internal/server/server.go |
| The GitLab API endpoint patterns and its fallback | internal/gitlabapi/handler.go |
| The management API endpoint patterns | internal/managementapi/binding.go |
| The GitLab API mount | cmd/artifact-registry/wire_gitlab.go |
The OCI /v2/ mount |
cmd/artifact-registry/wire_oci.go |
| The Maven pattern on the root dispatcher mux | cmd/artifact-registry/wire_root_dispatcher.go |
| The npm route patterns, when authorization is not enforced | cmd/artifact-registry/wire_npm.go, wire_root_dispatcher.go |
The root dispatcher /, when authorization is enforced |
cmd/artifact-registry/wire_root_dispatcher.go |
Specific behavior:
- The probe endpoints are served outside
HTTPServerMiddleware, so they get nohttp_*series. - Under enforced authorization,
withNpmAuthzclones the request before the npm mux runs, so every npm request books the root dispatcher/. The npm route patterns are therefore seeded only when authorization is not enforced. - The management API mount
/api/v1/gets no seeded series. Its handler gives the same request to an inner mux, and that mux writes its own pattern on every request, so/api/v1/never reaches theroutelabel. For the same reason, the root dispatcher/is seeded only when authorization is enforced. npm.RouteRegistration.Patternreturns the full pattern that the npm mux registers, so the seed and the mux use one source.
docs/dev/observability.md replaces the old "Do not seed a zero child" rule with the startup policy, and lists the HTTP families that get series at startup.
Deviation from the acceptance list.
Item 1 of the issue says that the composition root calls the helpers.
Three of the Handle calls are in library packages: internal/server, internal/gitlabapi and internal/managementapi.
These packages own nested mux patterns that reach the route label, so the seed calls stay beside the registration of each pattern.
The operator amended item 1 on the issue to this shape.
The stack
| Part | MR | Scope |
|---|---|---|
| 1 | This MR | HTTP server series |
| 2 | !3354 (merged) | Datastore query series |
| 3 | !3355 (merged) | Storage query series |
| 4 | !3348 (merged) | Job series |
Merge the parts in order.
Each part targets the branch of the part before it.
When all four parts merge, the tree is the same as the !3348 (merged) head before the split, 9f5b5285608815c320547be1552405c410687eb9.
Spec coverage
Source: the issue card acceptance list. No spec governs startup series creation; S03-B and S27 govern only the label values.
| # | Criterion | Tests |
|---|---|---|
| AC-1 | The owning package exposes an Init helper; the calls sit beside the registration of the pattern | TestInitHTTPRoute_CreatesTheRouteChildrenWithoutAnObservation, TestWiring_SeedsHTTPSeriesWithNoTraffic, TestIntegration_SeedsMountedRoutesAtStartup |
| AC-2 | Histogram children are created without an observation | TestInitHTTPRoute_CreatesTheRouteChildrenWithoutAnObservation (count, sum and every bucket) |
| AC-4 | HTTP seeds come from the mounted mux patterns; the probe endpoints are not seeded | TestWiring_SeedsHTTPSeriesWithNoTraffic, TestIntegration_SeedsMountedRoutesAtStartup (one request per mount, label read back from the live chain), TestNpmSeedPatterns_SeedThePatternTheExitPathReads, TestMountSlugAnchoredFormats_SeedsRootMountUnderTheEnforcingPosture, TestRouteTable_MatchesBuiltMux |
| AC-7 | The HTTP series are on /-/metrics of a fresh instance with no traffic |
TestWiring_MetricsEndpointServesSeededSeriesWithNoTraffic |
| AC-8 | AuditCardinality passes with the seeded series present |
TestWiring_CardinalityAuditPasses, TestIntegration_SeedsMountedRoutesAtStartup |
| AC-10 | docs/dev/observability.md states which families exist at startup |
Documentation, not a test row. |
The issue card lists no error case and no security consideration. Tenant labels stay unseeded by construction: every seeded value comes from code.
E2E scenario catalog
This MR changes one row in docs/testing/e2e/management.md and adds no row.
The row e2e.management.observability.metric-families-registered still passes, because it asserts a subset of families and a new series breaks nothing.
Its explanation now applies only to the families that startup does not seed.
Size
The diff is 571 added and 32 removed lines in 18 files, measured with git diff origin/main...HEAD --numstat.
| File group | Files | Added | Removed |
|---|---|---|---|
Go tests (*_test.go) |
7 | 457 | 19 |
| Production Go | 9 | 91 | 6 |
docs/ |
2 | 23 | 7 |
The production change is 91 lines. The tests are most of the diff, and they cover one family, so a further split does not make the review smaller.
No customer-visible surface changes, so docs/user/ is not affected.
Related to #1595 (closed)