Queue Zoekt indexing when project code search falls back

What does this MR do and why?

A project with no Zoekt repository in a Zoekt-enabled namespace falls back to advanced search for project-level code search, and nothing ever queues its indexing. This MR queues it from the fallback, behind zoekt_index_project_on_search_fallback (gitlab_com_derisk, off by default; off keeps today's behavior).

The existing trigger in Search::Zoekt::SearchResults#empty_results_preflight_check? (index_async when no node has the project) can't fire for these projects:

  1. EE::Search::ProjectService#zoekt_nodes is Node.searchable_for_project(project), which joins zoekt_repositories. With no repository it's empty.
  2. So zoekt_node_available_for_search? and use_zoekt? are false, search_type returns advanced, and Search::Zoekt::SearchResults is never built.
  3. Forcing search_type=zoekt fails zoekt_search_type_errors ("Exact code search is not available") for the same reason.

Such a project is indexed only by a push to its default branch, an import, or a transfer, so one created after its namespace was rolled out and not pushed to since stays unindexed.

With the flag on, when zoekt_node_available_for_search? is false, the service queues Search::Zoekt.index_async(project.id) if:

  • at least one replica of the root namespace is ready,
  • the project has no Zoekt repository, and
  • its repository isn't empty.

The result is memoized per service instance, and IndexingTaskWorker deduplicates on project and task type (until_executed). IndexingTaskService creates the zoekt_repositories row before indexing starts, so the first job that runs stops further queueing, even if indexing then fails.

Each project whose job was actually queued logs Queued indexing for project without Zoekt repository on search fallback (json.subcomponent: zoekt, with gl_project_id), charted in the "Zoekt: Indexing queued from project search fallback" panel on the Zoekt: Health Dashboard. Nothing is logged when index_async returns false (Zoekt indexing disabled) or the job is deduplicated.

Why only once a replica is ready
  • A replica is ready only when all its indices are (ReplicaStateService), so initial indexing is done and a project still without a repository was missed.
  • Before that, InitialIndexingEventWorker creates repositories for every project in each index's project_namespace_id range, so the project gets one anyway.
  • Queueing earlier could duplicate it: with no repository in the replica, Router.fetch_indices_for_indexing picks the index with the most free storage, not the one whose range covers the project, and (zoekt_index_id, project_id) uniqueness doesn't stop a second row on another index.
  • Follow-up, not in this MR: Router iterates every replica, including pending ones, so a namespace with several replicas can still get a repository on a pending replica. GitLab.com uses one replica by default.
Evidence from GitLab.com (Kibana and a Database Lab clone, 2026-10-29)
  • In one large Zoekt-enabled root namespace with a ready replica, 1,518 of 100,145 projects have no Zoekt repository, 583 of them with a non-empty repository.
  • Project-level blob searches for such projects went to advanced search, with no IndexingTaskWorker job for them over the same three days.
  • Group-level searches in the same namespace all used Zoekt.

Aggregate figures only; no namespace or project identifiers.

References

Screenshots or screen recordings

Not applicable: backend only, no UI change.

Database

No database review needed: two single-row existence checks, each served by an existing index. They run only on the fallback path, with the flag on, and only when the namespace is searchable with Zoekt. The result is memoized per request, and the repository check is skipped when no replica is ready.

SELECT 1 AS one FROM "zoekt_replicas" WHERE "zoekt_replicas"."namespace_id" = <root_namespace_id> AND "zoekt_replicas"."state" = 10 LIMIT 1
SELECT 1 AS one FROM "zoekt_repositories" WHERE "zoekt_repositories"."project_identifier" = <project_id> LIMIT 1
Worst-case timings (Database Lab clone of gitlab-production-main, 2026-10-29 08:22 snapshot)

zoekt_replicas has ~158,000 rows and zoekt_repositories ~6.3 million. No namespace has more than 2 replicas and no project more than 2 repositories, so neither lookup reads more than 2 rows.

Query Case Plan Buffers Execution
Ready replica Namespace with the most replicas (2), match Index Scan on index_zoekt_replicas_on_namespace_id_enabled_namespace_id 7 hit 0.09 ms
Ready replica Miss: only a pending replica Same, 1 row removed by state filter 7 hit 0.14 ms
Repository Project with the most repositories (2) Index Only Scan on idx_zoekt_repositories_project_state_schema 8 hit 0.13 ms
Repository Miss: project with no repository Same, 0 heap fetches 5 hit, 1 read 1.35 ms

With a cold cache, earlier runs took ~3 ms each, reading 3 to 4 pages.

How to set up and validate locally

  1. Run bin/rspec ee/spec/services/ee/search/project_service_spec.rb -e "when searching with Zoekt".
  2. The when no Zoekt node has the project context covers queueing, the log entry, no log when nothing was queued, no ready replica, an existing repository, an empty repository, and the flag off.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code

Edited by Ravi Kumar

Merge request reports

Loading
Loading