<!--IssueSummary start--> <details> <summary> Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards. </summary> - [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=624419) </details> <!--IssueSummary end--> ### Release notes GitLab Exact Code Search (powered by Zoekt) now scales to large, deeply nested enterprise code corpora. Users can scope searches to any point in the group hierarchy without losing context, see actionable syntax errors inline, export result sets for downstream analysis, control indexing behavior for large files, and query exact code search programmatically through `glab` and MCP for use in agentic development workflows. ### Problem to solve As a **developer or application security engineer at a large enterprise with tens of thousands of repositories**, I want to search all of my organization's code accurately, understand when my results are incomplete, and pull those results into my own tooling, so that I can trust GitLab Exact Code Search as a replacement for a dedicated code search product. [Trigram indexing](https://about.gitlab.com/blog/exact-code-search-find-code-faster-across-repositories/#positional-trigrams-the-secret-to-lightning-fast-exact-matches) has been a significant improvement but the current solution does not work at enterprise scale. ##### Environment to Consider: Consider an environment representative of our largest self-managed and SaaS customers: - 15,000+ repositories under a single top-level group (and oftentimes across many top-level groups) - Most repositories sit **5 levels deep** to organize work around organizational structures. - Legitimate, security-driven searches routinely return **more than 10,000 matches** At this scale, five categories of problems make the feature unusable: 1. **Scope selection doesn't match how large hierarchies are navigated.** The dominant use case is "search everything from the top-level group," but the UI makes that hard to reach and easy to accidentally narrow. 2. **Errors are silent or unrecoverable.** Search syntax errors are returned by the API but never surfaced in the UI. Backend connection failures leave the search box in a state that cannot be resubmitted. 3. **Users cannot tell when results are incomplete.** Files skipped due to Zoekt's size and trigram limits are invisible, and the result set is capped (this cap is at the very least hard-coded for SaaS). 4. **No export path.** A 1,000+ result security sweep cannot be exported, deduplicated by repository, or divided among reviewers. 5. **No programmatic access.** There is no MCP server and no `glab` support for exact code search, which is a hard blocker for agentic coding workflows. ### Intended users - [Sasha (Software Developer)](https://handbook.gitlab.com/handbook/product/personas/#sasha-software-developer) — primary; day-to-day code navigation and discovery - [Delaney (Development Team Lead)](https://handbook.gitlab.com/handbook/product/personas/#delaney-development-team-lead) — assessing blast radius of a change across services - [Amy (Application Security Engineer)](https://handbook.gitlab.com/handbook/product/personas/#amy-application-security-engineer) — primary for export and completeness; CVE exposure sweeps across the full corpus - [Priyanka (Platform Engineer)](https://handbook.gitlab.com/handbook/product/personas/#priyanka-platform-engineer) — indexing configuration and operational visibility - [Dakota (Application Development Director)](https://handbook.gitlab.com/handbook/product/personas/#dakota-application-development-director) — economic buyer for a tooling consolidation decision ### User experience goal A user in a corpus of any size should be able to: - Search exact code from any level of the group hierarchy, with their current scope clearly shown and easily widened to the top-level group - Understand immediately and in the UI why a query failed and how to fix it - Know whether the result set they are looking at is complete or not - Export a result set for analysis outside GitLab - Run the same query from the CLI or from an agent _without_ provisioning a Personal Access Token ### Proposal #### 1. Scoping and search-mode UX at scale - **Make the current group the searchable scope.** When a user is already viewing a group, that group should be selectable (and ideally pre-selected) as the search scope. Today it is not an option in the combo box, so users cannot search where they already are. - **Fix project → group coupling.** Selecting a specific project silently resets **Group** to that project's immediate parent, changing the user's scope without their intent. - **Suppress or explain the branch selector on non-default branches.** After narrowing to a project, the UI offers a Git revision dropdown. Selecting any non-default branch produces: &gt; Exact code search (powered by Zoekt) is disabled since `&lt;branch&gt;` is not the default branch. Offering a control that is guaranteed to break a capability is a UX defect independent of the underlying single-branch limitation. - **Provide a direct path to top-level-group code search.** Customers rely heavily on bookmarks as a workaround to land directly on code search scoped to the top-level group. A first-class entry point should exist. - **Fix wildcard and empty-query behavior.** `*` is injected as a default term and is correct for Projects, Work Items, and Merge Requests, but in Code it is a literal `*` search — while still showing a result count that implies otherwise. Removing the `*` then executes a search on an empty string and renders a "No results found" state rather than an empty state. - **Do not search on every keystroke.** Exact code search re-queries per keypress (maybe as a side effect of the left-nav scope counts refreshing). This produces errors on unclosed quotes and partial filters, and generates enough load to trigger backend failures. Debounce, or require explicit submission for the code search tab. - **Invert the `Exclude forks` / `Include archived` defaults for private projects/repos.** The current polarity is tuned for public code search (where forks are prevalent). For our customer base, excluding forks should be the sensible default. #### 2. Error surfacing and recovery - **Render Zoekt syntax errors in the UI.** The API already returns actionable text; it is only reachable through the browser's network debugging tab. Examples the customer captured: &gt; `bad request: error building search request: failed to convert raw request to SearchRequest: failed to convert and child 0 to proto: failed to parse search query: query: unknown case argument "", want {yes,no,auto}` &gt; `bad request: error building search request: failed to convert raw request to SearchRequest: failed to convert and child 0 to proto: failed to parse search query: query: unterminated quoted string` These should be translated into user-facing guidance, not swallowed behind a generic "A problem has occurred." - **Make failed searches resubmittable.** When the Zoekt backend cannot be contacted (`ClientConnectionError` on `blobSearch`), the only way to retry is to mutate the query and change it back. A retry affordance is needed. - **Distinguish backend failure from zero results.** "Cannot connect to the Zoekt" currently renders in the same visual frame as a legitimate empty result set. #### 3. Result completeness and export Maybe best suited for the ~"devops::production engineering" team as it could be placed within our ~"Category:Rate Limiting Framework" - **Expose which files were skipped during indexing.** Files over 1 MB or exceeding 20,000 trigrams are silently excluded (see [issue 455073](https://gitlab.com/gitlab-org/gitlab/-/issues/455073)). For example, `package-lock.json` can easily reach these limits which means these files are not indexed, giving customers an **incomplete view of their CVE exposure**. Silent incompleteness during a security investigation is the most severe issue in this report. (maybe there's a different solution here) - **Allow overriding indexing limits, inheritable from a top-level group.** Zoekt supports this today via the `LargeFiles` option ([builder.go#L98](https://github.com/sourcegraph/zoekt/blob/main/index/builder.go#L98)). This would allow our customers to force-index `yarn.lock` and `package-lock.json`. Setting it at the top-level group and having is cascade down would be fantastic. - **Raise or paginate past the ~5,000 result cap.** "Refine your search" is not a valid answer for a corpus of this size; the customer has legitimate result sets exceeding 10,000 entries. - **Add result export.** CSV/JSON export of matches, so results can be deduplicated by repository, checked programmatically, or split across multiple reviewers. This is explicitly framed as a security workflow requirement, not a convenience. - **Optional: allow additional branches to be indexed.** Customer teams would benefit from targeting extra branches, configurable at group or project level. Tracked at [issue 403307](https://gitlab.com/gitlab-org/gitlab/-/issues/403307). #### 4. Programmatic and agentic access - **OAuth Support**: reuse the existing `glab` OAuth connection so developers do not have to create Personal Access Tokens or register a new OAuth app. - **Add exact code search to the GitLab MCP server.** Called out as "a big issue with the advent of LLMs and agentic coding workflows." An agent that cannot search the organization's code exactly cannot ground its work in that code. **Potential Workaround:** Accessible via the search API with `search_type=zoekt`, callable by LLMs through the MCP `gitlab_search` tool. - **Add exact code search to `glab`.** Hacky but you can use a GraphQL query and it works for simple cases and breaks quickly beyond them. --- ### Further details This should be considered an opportunity for competitive displacement. Zoekt already clears the bar but our gap is entirely in scoping UX, error handling, completeness guarantees, export, and programmatic access. ##### GitLab's Search Mechanisms **1. Global/Advanced Search (Elasticsearch / OpenSearch)** This is the traditional keyword-based search powering the main GitLab search bar. It indexes issues, MRs, commits, wikis, users, milestones, and code across the entire instance. The tech stack is **Elasticsearch or AWS OpenSearch** (Lucene-based), using inverted indexes, tokenizers, and analyzers for full-text search. It supports boolean operators and field-level filtering. On GitLab.com, this is always on for paid tiers. **2. Exact Code Search (Zoekt)** A separate layer specifically for code, powered by **Zoekt** (an open-source code search engine). It handles exact match and regex modes across code files at the group or project level. Zoekt does _NOT_ replace Elasticsearch as it only handles code scope; all other scopes (issues, MRs, etc.) still go through Elasticsearch. On GitLab.com, it's enabled by default for paid subscriptions. **3. Semantic Code Search (Vector Embeddings)** The newest system, built on the **Active Context framework** (`gitlab-active-context` gem). Semantic Code Search converts code into vector embeddings using `text-embedding-005` (Gemini) on SaaS, then stores them in a vector store (Elasticsearch, OpenSearch, or PostgreSQL with pgvector). Queries are also embedded and compared via k-nearest-neighbor (kNN) similarity -- finding code by _meaning_, not keywords. ###### LLM Accessibility | System | LLM Accessible? | How | |---|---|---| | **Global/Advanced Search** | **Indirectly** | No dedicated MCP tool, but the GitLab MCP server exposes a `gitlab_search` tool that wraps the REST search API, which LLMs can call. | | **Exact Code Search (Zoekt)** | **Indirectly** | Same as above. Accessible via the search API with `search_type=zoekt`, callable by LLMs through the MCP `gitlab_search` tool. | | **Semantic Code Search** | **Yes, natively** | Exposed as an MCP tool (`semantic_code_search`) for Duo Chat, Duo Agent Platform, and external AI platforms. Also available via REST API (`GET /api/v4/projects/:id/search/semantic`) and `glab` CLI. | ### Permissions and Security - **Export** is the primary area needing review. Exported results must respect the requesting user's project-level access; a result set spanning thousands of repositories must not become a path to leaking code from projects the user cannot read. Rate limiting and audit logging of large exports should be considered. - **Indexing overrides** (`LargeFiles`) should require Maintainer or Owner at the group where the setting is applied, since it changes indexing cost and behavior for every project underneath. - **Error message surfacing** must expose parser errors only — care is needed not to leak backend topology, host names, or internal service details in the `ClientConnectionError` path. - Expected impact by role: - [ ] No access (0) — no change - [ ] Guest (10) — [TBD by product] - [ ] Reporter (20) — [TBD by product] - [ ] Developer (30) — search and export within permitted scope - [ ] Maintainer (40) — project-level indexing configuration - [ ] Owner (50) — group-level inheritable indexing configuration - Recommend a threat model review of the export path per the [Threat Modeling handbook](https://about.gitlab.com/handbook/security/threat_modeling/#threat-modeling). ### Documentation **Add an advanced search syntax section.** Current examples are simple enough to be misleading about real query patterns. See support request [739112](https://support.gitlab.com/hc/en-us/requests/739112). Best case, a user hits an error and has to reverse-engineer the fix. Worst case, they get incomplete results and never know. The advanced section should show the same query across different search modes and document error recovery. **Document operational behavior**, none of which is currently discoverable: 1. How often repositories are re-indexed 2. How to determine when a given repository was last indexed 3. How to force a re-index of a repository **Address the SaaS gap in the Zoekt docs.** [docs.gitlab.com/integration/zoekt/](https://docs.gitlab.com/integration/zoekt/) covers several of these limits and operational tasks, but the settings and rake tasks described there are unavailable to SaaS customers. The page should state that clearly and point SaaS users to whatever equivalent exists. In addition: - Update [Exact code search](https://docs.gitlab.com/user/search/exact_code_search/) with the advanced syntax section and error recovery guidance - Update the [known issues](https://docs.gitlab.com/user/search/exact_code_search/#known-issues) section as limits are lifted - Update [Zoekt integration](https://docs.gitlab.com/integration/zoekt/) to distinguish self-managed from SaaS capability - Document any new export, `glab`, or MCP surface - Update [permissions](https://docs.gitlab.com/user/permissions/) if indexing configuration introduces new role requirements ### Availability & Testing - **Availability risk is concentrated in the keystroke-search fix and the result cap.** Debouncing should *reduce* Zoekt load. Raising the 5,000 result ceiling and adding export will *increase* it, and both need load testing against large volumes of repositories before rollout. - Export should be treated as a potentially long-running operation — background job with a download artifact rather than a synchronous response. - Test coverage needed: - **Unit:** query parser error translation into user-facing messages; scope resolution when navigating from group to project and back - **Integration:** Zoekt backend failure and retry paths; `LargeFiles` override inheritance from top-level group down through 7 levels of nesting; export permission filtering - **End-to-end:** search from a deeply nested group; scope persistence across navigation; export a large result set; failed-search resubmission - **Cross-browser:** yes, for the search input debouncing and error rendering changes - Reach out to the counterpart SET via [Quad Planning](https://handbook.gitlab.com/handbook/engineering/infrastructure/test-platform/quad-planning/). ### Available Tier Exact code search is currently Premium and Ultimate. Recommend these enhancements follow the existing tiering, with a possible exception: - **Free/Premium/Ultimate (parity with today):** scoping UX fixes, error surfacing, documentation, retry behavior, fork/archive defaults - **Ultimate (candidate for differentiation):** result export, indexing limit overrides and skipped-file visibility, raised result caps We could potentially think about this in light of ~"Category:Rate Limiting Framework" Rationale: the UX and error-handling items are defect-class and should not be gated. The export and completeness items map directly to security workflows and to the Ultimate buyer. ### Feature Usage Metrics Instrument via [internal events](https://docs.gitlab.com/development/internal_analytics/internal_event_instrumentation/quick_start/): - Exact code searches executed, segmented by scope level (instance / top-level group / nested group / project) - Search error rate, split by syntax error vs. backend connection failure - Rate of searches abandoned immediately following an error (proxy for unrecoverable state) - Searches returning a truncated result set (hitting the cap) - Export invocations, and result-set size distribution at export - Queries issued via `glab` and via MCP, as a share of total - Repositories with an active `LargeFiles` indexing override - Views of the "skipped files" surface ### What does success look like, and how can we measure that? **Business outcomes** - Accounts convert to GitLab Exact Code Search form competitors - Growth in exact code search usage at accounts with &gt;5,000 repositories - Exact code search queries appear as a meaningful share of agentic/Duo tool calls **Acceptance criteria** - A user viewing any group can search exact code scoped to that group, and the scope shown matches the scope searched - Selecting a project does not silently alter the group scope - Every failed search renders a message stating what is wrong with the query - A failed search can be resubmitted unchanged - A user can determine which files in a given scope were excluded from indexing and why - A group Owner can set a `LargeFiles` override that is inherited by all descendant groups and projects - A result set larger than the display cap can be exported in full, respecting the requesting user's permissions - The same query can be executed from the UI, from `glab`, and from an MCP client, returning equivalent results, with `glab` reusing existing OAuth credentials ### What is the type of buyer? Primarily the **Development Director / VP Engineering** consolidating developer tooling spend, with a strong secondary in **Application Security** for the completeness and export capabilities. In enterprise accounts the purchase decision is frequently a displacement decision — the question is not "should we buy code search" but "can we stop paying for the code search we already have." ### Is this a cross-stage feature? Yes. Expected to touch: - **Global Search** (owning group) - **Foundations / UX** — search scope selection, error states, empty states - **Duo / AI-powered stages** — MCP surface for exact code search - **Editor Extensions / `glab`** — CLI subcommand and OAuth reuse - **Secure** — the CVE-exposure use case that motivates completeness and export - **Documentation** — advanced syntax and operational guidance Recommend looping in the PM and Product Designer for each before breakdown. ### What is the competitive advantage or differentiation for this feature? GitLab's structural advantage is that code search lives inside the platform that already holds the code, the permissions model, the CI results, and the AI agents. A developer searching in GitLab is one click from the blame view, the MR, and the pipeline. An agent searching through GitLab MCP is already authenticated as the user with the correct access scope — no separate index, no separate license, no separate credential. That advantage only pays off if the search itself is trustworthy at enterprise scale. Right now the trigram index is competitive and the experience around it is not. ### Links / references