## Problem Today we use a single index for all data types. There are many problems with this described in https://gitlab.com/gitlab-org/gitlab/-/issues/3217 but a major motivating factor for moving to indices for each data type is performance. Based on https://gitlab.com/gitlab-org/gitlab/-/issues/3217#note_399228783 we can reduce the total response time of searches for issues by `54%` if those issues are in a dedicated index rather than the combined index with all the other data types. We will likely see similar benefits for other data types but we should just start with 1 data type for now to iterate and learn at each point. ## Solution Move the issues to a dedicated index. This will have several challenges that we need to overcome and may require several MRs to accomplish. The challenges will be: 1. Migrating data so that there is no downtime in issue search. We may wish to pause indexing of issues during the migration so that the new issue index is fully populated with the new data before we index the updates. We may use some of the discussion in https://gitlab.com/gitlab-org/gitlab/-/issues/234046 to figure out what is a suitable migration strategy. I happen to know from [previous testing](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/2530#note_398522652) that migrating only the issues takes less than 2mins with a reindex so this may be totally acceptable to ship to customers as a transparent background migration assuming there aren't likely self-managed customers that have much larger indices than use 2. Issue search relies today on using joins to the parent project object for permissions checks. We'll need to denormalize our data as described in https://gitlab.com/groups/gitlab-org/-/epics/2054 before we can search issues in a dedicated index ### Steps 1. Denormalise the issue permission data per https://gitlab.com/groups/gitlab-org/-/epics/2054 . This may be done over a few MRs: 1. I think we can probably handle the project and group wide searches avoiding the need for joins at all based on the inline permission (ie. just `confidential` field) and the existing `project_id` field already on the issue. 1. Next we need to denormalise the project issue visibility level and project visibility level (possibly this can actually just be a single field depending on the way it's used) onto the issue type. This requires a migration to update all the existing issues with this field. It also requires triggering an issue update whenever the project permissions change. 1. Change global searches to use the new denormalized fields rather than the join to parent 1. Add support for setting the `index` used when indexing based on the record type (the bulk indexers all hardcode `gitlab-<ENV>` that will need to change. We can still use the single bulk indexer since bulk indexing can support to writing to multiple indices. 1. Add a migration that pauses indexing, moves the issues over to the new index, ensures reads and writes for issues are now coming from the new index and then deletes the issues from the existing index ## Proposal (issues will be opened for each) These should be done in order unless otherwise noted 1. Remove joins from the Elasticsearch query for project/group scoped issues search 1. De-normalize the issue permission data. <-- will include migration and can be done in multiple MRs 1. Remove joins from the Elasticsearch query for globally scoped issues search 1. Migration to copy issue data to new index 1. Migration to remove issue data from combined index ** 1. Rake task changes to support new index ** 1. Backend changes for supporting new indexes during Zero Downtime Re-indexing ** 1. Admin UI changes to support new index (any time after new index created/data migrated to it) ** must be released in conjunction with the `Migration to copy issue data to new index`