Queue Zoekt indexing when project code search falls back
What does this MR do and why?
A project with no Zoekt repository in a Zoekt-enabled namespace falls back to advanced search for project-level code search, and nothing ever queues its indexing. This MR queues it from the fallback, behind zoekt_index_project_on_search_fallback (gitlab_com_derisk, off by default; off keeps today's behavior).
The existing trigger in Search::Zoekt::SearchResults#empty_results_preflight_check? (index_async when no node has the project) can't fire for these projects:
EE::Search::ProjectService#zoekt_nodesisNode.searchable_for_project(project), which joinszoekt_repositories. With no repository it's empty.- So
zoekt_node_available_for_search?anduse_zoekt?are false,search_typereturnsadvanced, andSearch::Zoekt::SearchResultsis never built. - Forcing
search_type=zoektfailszoekt_search_type_errors("Exact code search is not available") for the same reason.
Such a project is indexed only by a push to its default branch, an import, or a transfer, so one created after its namespace was rolled out and not pushed to since stays unindexed.
With the flag on, when zoekt_node_available_for_search? is false, the service queues Search::Zoekt.index_async(project.id) if:
- at least one replica of the root namespace is
ready, - the project has no Zoekt repository, and
- its repository isn't empty.
The result is memoized per service instance, and IndexingTaskWorker deduplicates on project and task type (until_executed). IndexingTaskService creates the zoekt_repositories row before indexing starts, so the first job that runs stops further queueing, even if indexing then fails.
Each project whose job was actually queued logs Queued indexing for project without Zoekt repository on search fallback (json.subcomponent: zoekt, with gl_project_id), charted in the "Zoekt: Indexing queued from project search fallback" panel on the Zoekt: Health Dashboard. Nothing is logged when index_async returns false (Zoekt indexing disabled) or the job is deduplicated.
Why only once a replica is ready
- A replica is
readyonly when all its indices are (ReplicaStateService), so initial indexing is done and a project still without a repository was missed. - Before that,
InitialIndexingEventWorkercreates repositories for every project in each index'sproject_namespace_idrange, so the project gets one anyway. - Queueing earlier could duplicate it: with no repository in the replica,
Router.fetch_indices_for_indexingpicks the index with the most free storage, not the one whose range covers the project, and(zoekt_index_id, project_id)uniqueness doesn't stop a second row on another index. - Follow-up, not in this MR:
Routeriterates every replica, including pending ones, so a namespace with several replicas can still get a repository on a pending replica. GitLab.com uses one replica by default.
Evidence from GitLab.com (Kibana and a Database Lab clone, 2026-10-29)
- In one large Zoekt-enabled root namespace with a ready replica, 1,518 of 100,145 projects have no Zoekt repository, 583 of them with a non-empty repository.
- Project-level blob searches for such projects went to advanced search, with no
IndexingTaskWorkerjob for them over the same three days. - Group-level searches in the same namespace all used Zoekt.
Aggregate figures only; no namespace or project identifiers.
References
- Resolves #637026 (closed)
- Rollout: #637003
- Existing trigger last changed in ae5631d5
Screenshots or screen recordings
Not applicable: backend only, no UI change.
Database
No database review needed: two single-row existence checks, each served by an existing index. They run only on the fallback path, with the flag on, and only when the namespace is searchable with Zoekt. The result is memoized per request, and the repository check is skipped when no replica is ready.
SELECT 1 AS one FROM "zoekt_replicas" WHERE "zoekt_replicas"."namespace_id" = <root_namespace_id> AND "zoekt_replicas"."state" = 10 LIMIT 1
SELECT 1 AS one FROM "zoekt_repositories" WHERE "zoekt_repositories"."project_identifier" = <project_id> LIMIT 1Worst-case timings (Database Lab clone of gitlab-production-main, 2026-10-29 08:22 snapshot)
zoekt_replicas has ~158,000 rows and zoekt_repositories ~6.3 million. No namespace has more than 2 replicas and no project more than 2 repositories, so neither lookup reads more than 2 rows.
| Query | Case | Plan | Buffers | Execution |
|---|---|---|---|---|
| Ready replica | Namespace with the most replicas (2), match | Index Scan on index_zoekt_replicas_on_namespace_id_enabled_namespace_id |
7 hit | 0.09 ms |
| Ready replica | Miss: only a pending replica | Same, 1 row removed by state filter |
7 hit | 0.14 ms |
| Repository | Project with the most repositories (2) | Index Only Scan on idx_zoekt_repositories_project_state_schema |
8 hit | 0.13 ms |
| Repository | Miss: project with no repository | Same, 0 heap fetches | 5 hit, 1 read | 1.35 ms |
With a cold cache, earlier runs took ~3 ms each, reading 3 to 4 pages.
How to set up and validate locally
- Run
bin/rspec ee/spec/services/ee/search/project_service_spec.rb -e "when searching with Zoekt". - The
when no Zoekt node has the projectcontext covers queueing, the log entry, no log when nothing was queued, no ready replica, an existing repository, an empty repository, and the flag off.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.