Zoekt: Support offset-based pagination in coordinating node search API

Dmitry Gruzdrequested to merge
591915-zoekt-offset-pagination into master

What does this MR do and why?

Implements offset-based pagination for Zoekt standard search (non-multi-match), reducing payload size for deep pages.

Currently Rails requests up to 5,000 files from the coordinating node and paginates client-side. This MR changes the strategy:

  • Pages 1–10 (cache window): offset=0, limit=200 — same as before, preserving Search::Zoekt::Cache behaviour where 10 pages are fetched and cached in Redis in a single request
  • Pages 11+: offset=(page-1)*per_page, limit=per_page — single page fetched, coordinating node handles the offset

The coordinating node still collects and ranks the full result set across all shards; only the returned slice changes.

Activation conditions

The offset is only sent when both conditions are met:

  1. Feature flag zoekt_offset_pagination is enabled (default: false)
  2. All online Zoekt search nodes report version >= 1.9.0 via metadata['version']

The version check (Node.all_at_least_version?) handles all production version string formats:

  • "2026.02.27-v1.9.0-62-g6229205" (date + git suffix)
  • "v1.9.0" (plain with v-prefix)
  • "1.9.0" (plain)

Cache correctness

For pages beyond MAX_PAGES (10), the cache is bypassed entirely when offset pagination is active — a single-page response must not overwrite the batch cache entries written for pages 1–10. The result page index is also adjusted to 0 since Zoekt returns the slice starting at index 0, not page-1.

Changes

  • Search::Zoekt::Node: Add all_at_least_version?(version) class method
  • Search::Zoekt::Params: Add file_match_offset; update max_file_match_results for offset pagination
  • Search::Zoekt::SearchRequest: Include file_match_offset in payload when non-zero
  • Search::Zoekt::Cache: Bypass cache for deep pages when offset pagination is active
  • Search::Zoekt::SearchResults: Wire page and per_page through to Client → Params; use correct result page index for deep pages
  • GITLAB_ZOEKT_VERSION: Bumped to 1.9.0
  • Feature flag: zoekt_offset_pagination (gitlab_com_derisk, default: false)

How to set up and validate locally

  1. Ensure gitlab-zoekt >= 1.9.0 is running (see companion indexer MR: gitlab-zoekt-indexer!835 (merged))
  2. Enable the FF: Feature.enable(:zoekt_offset_pagination)
  3. Search for code at group level with many results, navigate past page 10
  4. Observe in logs/network that file_match_offset is included in the request payload for pages 11+, and pages 1–10 cache is unaffected

Related

Edited by Dmitry Gruzd

Merge request reports