## Problem GitLab search has two SLIs and both measure latency and errors. **Nothing measures search quality or per-surface usage.** A completed code audit of the monolith (2026-09-03, at commit `5eb84dc7132242e10adf9e5fce3d1a1a1de5b94b`) established the baseline below and found that the binding constraint is not a dark surface — it is the shape of the one event everybody relies on. `perform_search` carries no `additional_properties` at all (`config/events/perform_search.yml:5-6` lists only the `user` identifier). So even the surfaces that *are* instrumented cannot be segmented by scope, engine, search level or result count. Every per-surface and per-engine usage question is unanswerable **by construction**, not by omission. ## Coverage baseline as measured | Population | Emits a volume event | Notes | |---|---|---| | **Tier 1** — the 13 surfaces Global Search owns | **9 / 13 = 69%** | The ratchet: this is the population the group can be held to | | Tier 1, `perform_search` specifically | 7 / 13 = 54% | The stricter figure whenever the claim concerns `counts.all_searches` | | **Tier 1, SLI-only** | **4 / 13 = 31%** | **The most actionable number.** SLI recorded, no volume event — looks fully instrumented on a Grafana dashboard, absent from the analytics warehouse | | Tier 2 — 27 other retrieval surfaces | 2 / 27 = 7% | Cross-group ownership; a discovery surface, not a ratchet | | **Combined** | **11 / 40 = 28%** | 25 / 40 (62%) emit nothing whatsoever | The 31% SLI-only figure is the one to act on first, because those surfaces are *indistinguishable from working* on a dashboard. A reader cannot tell them apart from instrumented surfaces, which is why "search usage" numbers today are quietly wrong rather than visibly incomplete. ## Hard sequencing constraint **The first child must land before the second, third and fifth.** This is a constraint, not a preference. Adding surfaces to an event that carries no properties produces a **bigger number that is equally unusable**. Worse, it burns the Analytics Instrumentation review cycles that the schema change itself needs, and it makes the coverage ratchet move while analytical-question coverage stays at 2 of 10. Fix the schema, then widen the population. ## Children, in priority order Ranked by (analytical value unlocked) ÷ (implementation difficulty). | No. | Item | Why it is here | |---|---|---| | 1 | [Add `additional_properties` to the `perform_search` event](https://gitlab.com/gitlab-org/gitlab/-/work_items/627668) | One YAML plus two call sites unlocks every segmentation question at once. Hard prerequisite for 2, 3 and 5. | | 2 | [Instrument Zoekt code search volume exactly once across the controller and GraphQL layers](https://gitlab.com/gitlab-org/gitlab/-/work_items/627669) | Zoekt code search is measured by two layers that both fire on one page load; closing them separately double-counts. One item so one agent decides which layer owns the event. | | 3 | [Emit a zero-result signal on search events](https://gitlab.com/gitlab-org/gitlab/-/work_items/627670) | Zero-result rate is the only quality metric knowable synchronously on every engine. Deliberately a boolean, not a count — see below. | | 4 | [Emit the semantic search confidence value and add its missing SLI](https://gitlab.com/gitlab-org/gitlab/-/work_items/627671) | Semantic search already computes a relevance signal and discards it, and records no SLI at all. The cheapest real quality signal in the codebase. | | 5 | [Close the SLI-only gap on tab counts, autocomplete and aggregations](https://gitlab.com/gitlab-org/gitlab/-/work_items/627672) | Closes the 31% SLI-only gap — the surfaces that look instrumented and are not. | | 6 | [Investigate ownership and metric treatment of issuable list search (Tier 2)](https://gitlab.com/gitlab-org/gitlab/-/work_items/627673) | Investigation only. The most common search in the product is unmeasured, but ownership sits outside Global Search, so the deliverable is an ownership answer, not an MR. | ## Departures from the audit's own ranking The audit's §4 table was the input; these are the places this epic overrides it, with reasons. - **Its ranks 2 and 3 are merged into child 2.** The audit ranked the GraphQL `blobSearch` gap and the Zoekt web blobs gap separately, then noted in its own closure sketch that the Zoekt page fires *both* a controller render and a `blobSearch` call. Closing them independently double-counts every code search — a worse outcome than today's undercount, because it looks like growth. They are one decision and therefore one item. - **Its rank 4, "no result-count property", is narrowed to a zero-result boolean in child 3.** The audit established that basic search deliberately calls `without_count` (`lib/gitlab/search_results.rb:45`) and that Elasticsearch caps at `ELASTIC_COUNT_LIMIT = 10000` (`ee/lib/gitlab/elastic/search_results.rb:9`). The integer is therefore unavailable on one engine and censored on another, while the boolean is knowable everywhere. Same analytical value, lower difficulty, so a higher ratio than the audit assigned. - **Its rank 9, Duo `CodebaseSearch`, is folded into child 4 rather than tracked separately.** It calls the same retrieval query as the semantic REST endpoint. The transport-parity question should be answered once, by the agent already in that code. - **Its rank 8, MCP reconcilability, is not a work item.** It needs a shared key, which is exactly what child 1 plus the related MR below provide. It resolves as a consequence rather than as separate work. - **Its rank 11, Tier 2 issuable-list search, is kept despite a low ratio — but scoped as investigation (child 6).** Its low ratio is driven entirely by cross-group ownership, not by technical difficulty. An ownership answer is cheap; the implementation it might authorise is not, and is not in scope here. ## Non-goals Deliberately excluded. A stated non-goal is more useful to a future agent than a low-priority ticket. | Not doing | Why | |---|---| | The search request ID join key | Already in flight as [merge request 253076](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/253076), authored by the same person driving this epic. Duplicating it as a work item would fragment the effort. | | Adding `category` to the Service Ping leg | The audit rated this low ratio. `update_redis_values` runs before the Snowplow early return (`lib/gitlab/internal_events.rb:63` vs `:66`), so the web/API split exists in Snowplow only. Real, but low value against medium difficulty. | | Instrumenting the roughly 30 dark GraphQL search fields | Lowest ratio in the audit: many call sites, unclear ownership, and low analytical value per site. Tier 2 is a discovery surface, not a ratchet. | | A traffic-weighted coverage ratio | Not computable from static analysis. Needs per-endpoint request volume from Prometheus or Kibana, and the dark surfaces are precisely the ones with no counter to weight by. Stated as a gap rather than proxied. | ## Related work - [merge request 253076](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/253076) — "Add a search request ID to join searches to result clicks". Open, Draft, labelled for Analytics Instrumentation review. It also touches `config/events/perform_search.yml`, so **whoever picks up child 1 must coordinate with it to avoid a conflicting edit to the same YAML.** - The precedent that unblocks child 1: `resolve_feature_discovery_search` (`ee/app/services/ee/onboarding/feature_library/feature_match_service.rb:49-63`) is the only search event in the monolith shipping `additional_properties`. It is owned by a non-Global-Search team and has already passed Analytics Instrumentation review. The property gap is a convention problem, not a platform limitation. ## Done when All six children are closed, and the two questions "what is the volume of search per scope" and "what is the zero-result rate" are answerable from Snowplow without further instrumentation work.