Skip to content

Move projects to separate index

What does this MR do and why?

In this MR we move the projects to a separate elasticsearch index, to improve our performance.

Expected Run Time of migration: ~6 hours

Screenshots or screen recordings

Project Document Snippet on New Index

Screenshot_2023-06-05_at_2.04.20_PM

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

TEST INDEXING

  1. Make sure the elasticsearch is enabled. Navigate to the elasticsearch settings and make sure the two checkboxes Elasticsearch indexing and Search with Elasticsearch enabled under the Advanced Search are enabled.
  2. Complete the migration migrate_projects_to_separate_index
bundle exec rake gitlab:elastic:index
  1. Create a new Project, and verify that it gets indexed properly
curl -H 'Content-type: application/json' -XGET 'http://localhost:9200/gitlab-development-projects/_search' | json_pp
  1. Update the project description from the UI and verify again that description gets updated in the elastic by running the above command.
  2. Delete the project from the UI and verify that the documents get removed from the elastic by running the above command.

TEST SEARCHING

  1. Create a project with name new_project_index and description testing123
  2. Just for testing add the following line at the top in the method def migration_has_finished?(name). It will simulate as the migration migrate_projects_to_separate_index is not finished.
return false if name.eql?(:migrate_projects_to_separate_index)
  1. Now, create a project with a title old_project_index. Add the description same as you have given testing123`
  2. Now, search testing123. And notice that in the Project section you don't see the result new_project_index
  3. Now remove the line which you have added in the method def migration_has_finished?(name)
  4. Again, search testing123. And notice that in the project section you don't see the result old_project_index

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #344644 (closed)

Edited by Siddharth Dungarwal

Merge request reports