Copy issues to new index
We need to create a migration that will move all of the issue data from one index to the new issues index. The migration will use the existing Elasticsearch migration framework. Indexing should be paused during this migration.
A new issue config will be created to store the index settings/mappings (only relevant settings/mappings should be moved over from the combined index).
Issues search will need to handle the case where data may or may not be migrated. That can be done in a few ways. Two options I (and @mbergeron
though of):
- We will need to handle determining if the migration has run when doing Issues searches. This will tell if the combined or separated issue index should be used for the query. That could be done before each issues search by looking in the
_migrations
index and caching the ES migration status for subsequent calls. - Queries could be modified to search multiple indexes and aggregate on the _id field. (see https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-index.html#multi-index)
Epic order
- Remove joins from the Elasticsearch query for project/group scoped issues search
- De-normalize the issue permission data. <-- will include migration and can be done in multiple MRs
- Remove joins from the Elasticsearch query for globally scoped issues search
- Migration to copy issue data to new index
- Migration to remove issue data from combined index **
- Rake task changes to support new index **
- Backend changes for supporting new indexes during Zero Downtime Re-indexing **
- Admin UI changes to support new index (any time after new index created/data migrated to it)
** must be released in conjunction with the Migration to copy issue data to new index
Release Notes
Prior to this release, some Issue searches in Advanced Search would take a while to give results. This was because searching these results would have to reference the much larger index with everything in it.
We have split issues into their own index separate from the rest of the GitLab Index. This results in 50% faster Issue searching. This migration will happen automatically.
https://docs.gitlab.com/ee/user/search/advanced_global_search.html
TITLE: 50% Faster Issue Searching with Advanced Search