Skip to content

Add a migration to remove the wiki docs from the main index

Ravi Kumar requested to merge 336100-remove-wikis-from-main-index into master

What does this MR do and why?

Since wikis are now moved to a separate index, the wiki documents from the main index can be removed.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Make sure Elasticsearch is enabled

  1. Open the Rails console
bundle exec rails c
  1. Run the following command
require_relative 'ee/elastic/migrate/20230720010000_delete_wikis_from_original_index.rb' 
DeleteWikisFromOriginalIndex.new(20230720010000).migrate
  1. Now check the wiki docs on the main index. It should not have any docs
curl -XGET "http://localhost:9200/gitlab-development/_search" -H "kbn-xsrf: reporting" -H "Content-Type: application/json" -d'
{
  "query": {
    "bool": {
      "filter": [
        {
          "term": {
            "type": "wiki_blobs"
          }
        }
      ]
    }
  }
}' | json_pp

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Approximate time to completion

~ 3 hours

[6] pry(main)> number_of_documents = 605_735
=> 605735
[7] pry(main)> batch_size = 10000
=> 1000
[8] pry(main)> throttle_delay = 3.minutes
=> 3 minutes
[9] pry(main)> ((number_of_documents / batch_size) * throttle_delay)/60.minutes
=> 3

Related to #336100 (closed)

Edited by Ravi Kumar

Merge request reports