Skip to content

Project Wiki in separate index

Dmitry Gruzd requested to merge 336100-add-group-level-wiki-indexing into master

What does this MR do and why?

This MR migrates the wiki_blobs documents from the main index to the separate index. And newly created ProjectWiki gets indexed at this newly created index. The search query has been updated to use this new index. We are doing this because we want to move out the wiki_blobs from the main index to its own index for performance reasons.

Screenshots or screen recordings

Document snippet on the new index Screenshot_2023-04-18_at_10.32.16

How to set up and validate locally

TEST INDEXING

  1. Make sure the elasticsearch is enabled. Navigate to the elasticsearch settings and make sure the two checkboxes Elasticsearch indexing and Search with Elasticsearch enabled under the Advanced Search are enabled.
  2. Complete the migration migrate_wikis_to_separate_index
bundle exec rake gitlab:elastic:index
  1. Create a new Project Wiki, and verify that it gets indexed properly
curl -H 'Content-type: application/json' -XGET 'http://localhost:9200/gitlab-development-wikis/_search' | json_pp
  1. Update the wiki content from the UI and verify again that content gets updated in the elastic by running the above command.
  2. Delete the wiki from the UI and verify that the documents get removed from the elastic by running the above command.

TEST SEARCHING

  1. Create a wiki with title new_wiki_index and content testtest
  2. Just for testing add the following line at the top in the method def migration_has_finished?(name). It will simulate as the migration migrate_wikis_to_separate_index is not finished.
return false if name.eql?(:migrate_wikis_to_separate_index)
  1. Now, create a wiki with a title old_wiki_index. Add the content same as you have given testtest`
  2. Now, search testtest. And notice that in the Wiki section you don't see the result new_wiki_index
  3. Now remove the line which you have added in the method def migration_has_finished?(name)
  4. Again, search testtest. And notice that in the Wiki section you don't see the result old_wiki_index

NOTE: This MR implements only the ProjectWiki for the first iteration.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #336100 (closed)

Edited by Ravi Kumar

Merge request reports