Update supported vector stores to include Elasticsearch and OpenSearch
What does this MR do and why?
This MR adds Elasticsearch and OpenSearch to the supported technologies section of the GitLab AI docs.
Rationale
After extensive research and consideration of various options and tradeoffs, the decision has been made to store vector embeddings in Elasticsearch to power Retrieval Augmented Generation (RAG) and AI features in GitLab. Elasticsearch was chosen as the boring solution that meets GitLab's functional and technical requirements, aligning with our philosophy of selecting proven and reliable technologies. While other embedding storage solutions were considered, they did not fully meet GitLab's requirements.
More details around the proposal and decision are here (internal only).
As a consequence of this decision, Elasticsearch and OpenSearch will remain optional infrastructure for now. Elasticsearch will be used for all GitLab SaaS deployments, while self-managed users will need to bring their own Elasticsearch or OpenSearch cluster. For managed Elastic clusters, the Gold tier (the lowest tier) will be required.
OpenSearch, the open-source fork of Elasticsearch, is already supported for Advanced Search and it will remain supported as an open-source alternative to Elasticsearch for those who prefer a fully OSS solution.
Elasticsearch is already used at scale on GitLab.com, has been GA for self-managed for years, and is deeply integrated into the GitLab application. An ingestion framework is already in place to index new arbitrary data and generate embeddings. OpenSearch will be supported in GitLab's application code but not at at the SRE or production level by groupglobal search.
For more information on the impact to self-managed customers, see this doc (internal only).
The Global Search team has already begun shipping experimental AI features using Elasticsearch, highlighting the time to market benefits, and unlocking other AI features across the product. In the short-to-medium term, embeddings will be stored for a wide range of text documents, starting with issues, and including code, merge requests, projects, comments, epics, and milestones. The RAG foundation will be built to support hybrid search and retrieval for these embedded document types. Looking ahead, chat history can be stored in Elasticsearch to power agents and other AI-driven functionalities. RAG is expected to remain a valuable component of GitLab's solutions, even as context windows increase and inference costs decrease.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Required approvers
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
| Before | After |
|---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.