Name the Zoekt indexing task latency histogram for the quantity it measures

🤖 AI-authored change.

What does this MR do and why?

ZoektTasksSlis.increment_apdex reduced the measured task duration to a boolean before recording it, so no quantile and no threshold other than APDEX_THRESHOLD_S could be evaluated after the fact.

This observes the same duration into gitlab_search_zoekt_task_duration_seconds in the same call, with buckets 1, 5, 10, 30, 60, 120, 300, 600, 1800, 3600, 7200, 21600. The exact 60 edge answers a 60-second objective with a bucket read instead of a quantile interpolated across a bucket.

It also logs a Zoekt task Apdex SLI line with duration_s, target_s, success and the two labels, matching GlobalSearchSlis.record_apdex.

The observed duration is Time.current - task.perform_at, which covers queue wait and execution together — zoekt_tasks has no start timestamp — so the metric is named for that combined quantity, not for queue wait alone.

No new call site and no new clock. APDEX_THRESHOLD_S is unchanged at 7200; the histogram makes other thresholds queryable without moving the objective.

The docs carried a stale 30-minute Apdex threshold, corrected to the 7200s value the code compares.

References

Spec: specs/zoekt-task-sli/SPEC.md in gitlab-org/search-team/artifacts, merged as be17cc7aeea94bb82eb0c39829c28ca70506cd12, renamed to the _duration_seconds suffix in 69ef70966f2b9fd3d8e3104e07626fbe8b42124f (gitlab-org/search-team/artifacts!68 (merged)).

Screenshots or screen recordings

No user-facing change.

How to set up and validate locally

1. Enable Zoekt indexing and let an indexing task complete.
2. Scrape /-/metrics and confirm gitlab_search_zoekt_task_duration_seconds_bucket has a series at le="60".
3. Confirm a Zoekt task Apdex SLI line appears with duration_s set.

Verification

Exporter-side claims were proven against the prometheus-client-mmap 1.6.1 that Gemfile.lock:1814 pins, with the bucket constant parsed out of the shipped file rather than restated: an le series exists at exactly "60" and not at "50"/"75"; a 45s observation increments le="60" and every edge above but not le="30"; p50/p90/p99 all evaluate; an observation past the top edge lands in +Inf only. Five mutations each turned that proof red. Transcript: $6054462

ee/spec/lib/gitlab/metrics/zoekt_tasks_slis_spec.rb: 18 examples, 0 failures on the default seed and on 80085, 4242 and --order defined; rubocop clean on both changed Ruby files. Reverting the metric name fails 1 example, truncating the observed duration fails 2, truncating the logged duration fails 2, and moving the 7200 edge fails 1.

The buckets are integers because the exporter renders le as bucket.to_s, so a float edge would emit le="60.0".

Series cost: no new label dimension over the Apdex beside it. 112 node/task-type pairs report today, so ~1,680 new series fleet-wide (112 × 15 = 12 edges plus +Inf, _sum, _count); per process at most 59 pairs on any one of the 797 pods reporting, so ~885 worst case. The log line adds 18.6 lines/s against a Rails request rate of 34,700/s, ~0.05%, inside the 10% bar the structured-logging guidelines set for added log volume.

Correction to an earlier claim

Commit 3d58fc4f justified the _duration_seconds suffix by saying the old name "would be the only _latency_seconds metric name under app/, lib/ and ee/, against 56 distinct _duration_seconds names". Re-measured on master 116135f9fb2d, neither figure holds at that scope: under app/, lib/, ee/ there are 41 distinct _duration_seconds string literals, not 56, and only 11 are names actually passed to a histogram/counter/gauge declaration. The 56 reproduces only across the whole repository, and at that scope two _latency_seconds names already exist, so the "only one" half and the "56" half are true at different scopes and never together.

The narrow claim that does hold: _duration_seconds is the better-attested suffix for a registry-declared duration metric in this codebase, and both pre-existing _latency_seconds names (gitaly_health_check_latency_seconds, puma_check_latency_seconds) are health-check Gitlab::HealthChecks::Metric structs built by string interpolation on metric_prefix in SimpleAbstractCheck#metrics and GitalyCheck#metrics, not Prometheus registry metrics — which is also why a literal-only search reports zero of them. The suffix choice stands; the absolute count that was offered for it does not.

What I did not verify

• No production series. The rate and node-count figures come from the shipped Apdex counter; whether the new bucket series and its quantiles appear in gprd can only be checked after merge and deploy.
• The per-call cost figure is a microbenchmark outside Rails, so it is an upper bound and says nothing about contention under real Puma concurrency.
• gitlab-com/runbooks is untouched; the dashboard still plots only the Apdex ratio.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

---

🤖 Automated change — mention me to get a response. Feedback welcome. Ask to speak to John directly anytime.