Migrate blobs to dedicated Elasticsearch index
Proposal
For the many reasons described in #3217 (closed) and &2054 (closed) we want to move blobs to a dedicated Elasticsearch index.
Benefits
As described in &2054 (closed) and #3217 (closed) there are many performance gains we'll get from moving blobs to a separate index.
One major reason to start with blobs is that this is by far the largest part of our current index. So if we move that to a separate index then everything left in the current index will be considerably smaller and searches for those documents will be much faster.
Additionally once we have blobs in a separate index and a good migration strategy implemented #234046 (closed) we'll then be able to start implementing more advanced code search features like #223150 or #4175 (closed) .
Technical steps
- Write a migration to move the
blobsdata to a new index. Some ideas for this in #234046 (closed) - Search will need to search existing index as well as the new index during the migration process
- Later ship another migration that deletes
blobsfrom the original index
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.