Repository X-Ray build vector storage prototype using Elasticsearch
Background
As part of #437738 groupcode creation looks for suitable storage engine capable of storing and searching over embeddings vectors, to power semantic search capabilities. As part of broad platform wide effort to provide semantic search capabilities captured at !142787 (closed) it was suggested that groupcode creation use case can be used as a pilot use cases to use new solution
Goal
Build a prototype that will utilise Elasticsearch as vectorDB to index Projects::XrayReport
records. Then use this ES index to select 5 most relevant libraries available for given repository to be included to code generation prompt.
A mirror prototype was build for pg_vector
as vectorDB, which can be used as reference in this issue !142912 (closed)
Once prototype is build collect information about expected effort required to build and maintain production grade solution