Add a search request ID to join searches to result clicks

🤖 AI-authored change.

Adds search_request_id, a per-request UUID carried on both perform_search and click_search_result, so a result click can finally be joined to the search that produced it. Global search has two SLIs — an apdex and an error rate; nothing measures result quality, because no product-owned field survives on both events. This MR adds the join key and nothing else: no new events, no metrics, no query text.

Before	After
perform_search (backend) and click_search_result (frontend data attributes) share no field identifying the search request	Both carry the same per-request property = search_request_id (UUID), behind a flag
correlation_id looks like a join key but breaks on the POST /api/v4/usage_data/track_internal_event leg — the leg that feeds Service Ping — and is absent entirely for anonymous users	An explicit property survives both dispatch legs; correlation_id stays infra-owned
The property slot on click_search_result was declared "search term" and never populated by any of the 11 result partials	Redefined as the search request id; no historical data is lost
The identical tracking attribute hash was duplicated across all 11 result partials	One shared search_result_tracking_attrs helper
Self-Managed instances had no on-instance signal	The id is emitted into the structured request log as meta.search.request_id

• The id is minted in the controller, at the one place the event fires. increment_search_counters assigns @search_request_id, and a result link carries a join key only when that ivar is set — so no code path can render an id that no search emitted.
• Feature flag search_relevancy_join_key, gitlab_com_derisk, default off. With the flag off both events fire with byte-identical current payloads, and there is spec coverage asserting exactly that.
• CTR, MRR and abandonment become computable once the metrics land — this MR is only the prerequisite.
• No query text is emitted. Queries are user content, run to 4096 characters, and can contain confidential information.
• Covered scopes: issues, merge requests, notes, milestones, epics, wikis, users, commits, snippet titles, and blobs served by basic search or Elasticsearch.

Not covered (tracked in the groundwork item, not gaps in this MR): Zoekt-served blobs searches (they render a Vue mount point, emit no perform_search); anonymous clicks on Self-Managed (the click leg early-returns without gon.current_user_id, pre-existing dispatch behaviour); ranking beyond page one (click position is page-relative, so MRR is page-one scoped until page/per_page ship).

Related work items

• #627611 — parent: add relevancy metrics to global search.
• #627612 — groundwork: the remaining instrumentation the metrics need.
• #627613 — calculate and report the metrics.

Reviewer focus:

1. Analytics Instrumentation: is redefining the never-populated property field on click_search_result acceptable, or would you prefer a new custom property? The two Service Ping metrics referencing this event only count distinct users and neither reads property, so redefining it disturbs nothing. I took the built-in field because it is already wired through the frontend tracking utility, but I will switch if you would rather keep the built-in slot free.
2. Presence of the ivar is now the whole gate on the click side. search_result_tracking_attrs does not consult Feature — it attaches event_property iff @search_request_id is set. That is what makes an orphan join key unrepresentable, but it does mean the helper's behaviour is defined by controller state rather than by the flag. Is that the trade you want?
3. The view and helper specs set the ivar via assign(:search_request_id, ...). That is a truer unit boundary, but it means they do not exercise the flag check; the flag is covered in the controller spec.

---

🤖 Automated change. Mention @johnmason to leave feedback.

🤖 Mechanism, citations and verification (for agents / deep readers)

Mechanism

Citations are at head c77aaa89.

Minting the id. app/controllers/search_controller.rb:273-279 — increment_search_counters assigns @search_request_id = SecureRandom.uuid and emits additional_properties: { property: @search_request_id }; the flag-off branch emits the bare track_internal_event('perform_search', user: current_user) unchanged, so turning the flag off is a true rollback (property is optional in the definition). app/controllers/search_controller.rb:291-295 defines search_relevancy_join_key_enabled?, gated on Feature.current_request rather than current_user — anonymous search is supported, and a nil actor never matches a percentage-of-actors gate.

The click event. app/helpers/search_helper.rb:335-345 — search_result_tracking_attrs(position) returns event_tracking/event_label/event_value always and event_property only when @search_request_id is set. Rails' view_assigns copies the controller ivar into the view, so the event and all 11 result partials share one value per request. Each partial calls the helper — e.g. app/views/search/results/_note.html.haml:25. No frontend JS changes: the existing tracking utility already maps event_property to property.

Self-Managed logging. app/controllers/search_controller.rb:342 sets metadata['meta.search.request_id'] in search_payload_metadata, guarded on the ivar being present — count and aggregations reach that method without ever calling increment_search_counters, and get no id.

Event definitions. config/events/perform_search.yml:8-9 declares the new property; config/events/click_search_result.yml:14-15 redefines the existing never-populated one from "search term".

Documentation. doc/administration/logs/_index.md documents meta.search.request_id in the production_json.log section: what the id identifies, that the same value is the property on both events, and that the field is gated on the flag.

Flag definition. config/feature_flags/gitlab_com_derisk/search_relevancy_join_key.yml, default_enabled: false, milestone 19.4, rollout #627251.

Spec support. spec/support/helpers/search_result_tracking_helpers.rb:11-36 — tracked_link_properties(html) returns one entry per tracked link rather than a count, so a link that rendered the tracking attribute but lost only the property shows up as nil; capture_internal_events wraps (not replaces) Gitlab::InternalEvents.track_event so real definition validation still runs; emitted_join_key(events) reads the property off perform_search. Included for type: :view and type: :controller.

Why correlation_id was not usable

The client-side click_search_result does carry the search page's correlation id: app/views/layouts/_snowplow.html.haml:3-8 bakes the request context into window.gl.snowplowStandardContext (correlation_id from lib/gitlab/tracking/standard_context.rb:52) and get_standard_context.js:5 replays it verbatim at click time. On that leg the join already works. It breaks because InternalEvents.trackEvent fires twice (app/assets/javascripts/tracking/internal_events.js:42-52): once to Snowplow with the page context, and once via API.trackInternalEvent (app/assets/javascripts/api.js:972-993), which POSTs only project_id and namespace_id. lib/api/usage_data.rb then builds a fresh StandardContext with the POST's own correlation id — and that POST leg is the one running update_redis_values (lib/gitlab/internal_events.rb:129), so a correlation-id join could never reach Service Ping. It also early-returns for anonymous users (api.js:973-975). correlation_id is additionally an infra-owned request id echoed as X-Request-Id, so overloading it would couple a relevancy metric to request-id plumbing.

Verification

CI on the head pipeline is the authority: https://gitlab.com/gitlab-org/gitlab/-/pipelines/2838595416

What I did not verify

• Nothing was re-measured at head. The file:line citations above were read on branch jmason-search-relevancy-join-key at c77aaa89; the local spec and lint results predate the controller/helper refactor and are not a current pass/fail for the tree as it now stands.
• The rendered-request examples have no local verdict — this GDK returns prevent_cross_joins errors across the whole join-key block, including examples that predate this MR.
• No end-to-end join was performed against real Snowplow or Service Ping data — the equality of the two ids is asserted in-process by the controller spec, not observed downstream.
• The Self-Managed api_json.log shape for signed-in clicks is not asserted by any spec here; the id appears there as additional_properties inside params.