## Problem statement Currently we use Elasticsearch for code search within GitLab. Elasticsearch has turned out to be a poor fit for code search, and has resulted in some of our most highly requested improvements, primarily that code search returns less results than expected: https://gitlab.com/groups/gitlab-org/-/epics/8268. For example, currently we are unable to show multiple results in a given file in addition to being unable to do exhaustive and complete substring matching. This results in a code search experience which is difficult to rely on, as the results are incomplete. If a user needs to perform a follow up search using a different solution, they may as well start there to begin with. We believe this is eroding our adoption of search within GitLab, and a significant source of negative sentiment. ## Solution To solve these problems, we plan to swap out Elasticsearch for Zoekt. Zoekt will allow us to solve the problems outlined in https://gitlab.com/groups/gitlab-org/-/epics/8268, and provide search results a user can rely on. One of the main reasons why, is that Zoekt is purpose-built for code search, rather than Elasticsearch whose primary use cases lies outside of code. Note: For those who are not native Dutch speakers, the best pronunciation I could find is "zshukt" as per [this](https://en.wiktionary.org/wiki/zoekt) link. IPA(key): /zukt/ ### Iterations With a change of this magnitude, we will be taking an iterative approach to deployment. This will allow us to ensure the functionality is working, that it can scale, and that we have the operational expertise to meet our availability objectives. We plan to iterate in the following way: * Initial MVC * Enable for `/gitlab-cookbooks` namespace for small scale testing * Enable for `/gitlab-org` and it's associated public projects. This will allow us to dogfood internally, and avoid the need to solve all of the security challenges up front on initial release. (Code is always public, unlike issues/comments/etc which are indexed by elastic and thus out of scope) * Private Project support * Support for private projects * Indexing of all `gitlab-org` and `gitlab-com` projects * Completion of security reviews and any related findings * Opt-in Beta * First bundle Zoekt in Helm chart? Omnibus. * Support for flipping between Zoekt and Elasticsearch * Index customers who are willing to participate and have expressed an interest in code search * Opt-out beta * Percentage based deploy of Zoekt to namespaces to most of GitLab.com customers * Opt-out is available (see above) * Self-managed beta * Officially support deployments of Zoekt with Helm Chart and GitLab Operator * Add to reference architectures and all package types * Limited Availability * Support Zero-downtime reindexing * Support searching within large namespaces(over 30k projects) * Make sure code search API compatibility with Zoekt * GA * Multiple replica support * Remove flag to switch between Elasticsearch and Zoekt * Delete code search indexes from Elasticsearch > **Disclaimer**: We are currently enrolling customers to join the beta program on GitLab.com by request. We may not be able to accommodate all requests at this time. ## Feature flags * We have removed all feature flags.