Move projects to separate index
What does this MR do and why?
In this MR we move the projects to a separate elasticsearch index, to improve our performance.
Expected Run Time of migration: ~6 hours
Screenshots or screen recordings
Project Document Snippet on New Index
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
| Before | After |
|---|---|
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
TEST INDEXING
- Make sure the elasticsearch is enabled. Navigate to the elasticsearch settings and make sure the two checkboxes
Elasticsearch indexingandSearch with Elasticsearch enabledunder theAdvanced Searchare enabled. - Complete the migration
migrate_projects_to_separate_index
bundle exec rake gitlab:elastic:index
- Create a new Project, and verify that it gets indexed properly
curl -H 'Content-type: application/json' -XGET 'http://localhost:9200/gitlab-development-projects/_search' | json_pp
- Update the project description from the UI and verify again that
descriptiongets updated in the elastic by running the above command. - Delete the project from the UI and verify that the documents get removed from the elastic by running the above command.
TEST SEARCHING
- Create a project with
namenew_project_indexanddescriptiontesting123 - Just for testing add the following line at the top in the method
def migration_has_finished?(name). It will simulate as the migrationmigrate_projects_to_separate_indexis not finished.
return false if name.eql?(:migrate_projects_to_separate_index)
- Now, create a project with a title
old_project_index. Add thedescriptionsame as you have giventesting123` - Now, search
testing123. And notice that in theProjectsection you don't see the resultnew_project_index - Now remove the line which you have added in the method
def migration_has_finished?(name) - Again, search
testing123. And notice that in theprojectsection you don't see the resultold_project_index
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #344644 (closed)
Edited by Siddharth Dungarwal
