Skip to content

Remove hashed_root_namespace_id from users index documents

What does this MR do and why?

The field hashed_root_namespace_id has already been removed from the UserInstanceProxy but there are some documents that have this field. This migration will use the helper Elastic::MigrationRemoveFieldsHelper to remove this field from the documents.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  • Ensure elasticsearch is enabled

    Prepare the data

  • Run the following commands in the rails console

client = ::Gitlab::Search::Client.new
client.indices.put_mapping(index: User.__elasticsearch__.index_name, body: { properties: { hashed_root_namespace_id: { type: 'integer' } } })
client.update_by_query(index: User.__elasticsearch__.index_name, refresh: true, body: { script: { source: "ctx._source.hashed_root_namespace_id=1" } })
  • Now run the following command in rails console and ensure the count is greater than 0
client.count(index: User.__elasticsearch__.index_name, body: { query: { bool: { must: { exists: { field: 'hashed_root_namespace_id' } } } } })['count']
  • Now run the migration by the following command in the rails console
Elastic::DataMigrationService[20230928104245].send(:migration).migrate
  • Now ensure the count is 0 by running the following command in console
client.count(index: User.__elasticsearch__.index_name, body: { query: { bool: { must: { exists: { field: 'hashed_root_namespace_id' } } } } })['count']

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Run time

~ 20 hours

[3] pry(main)> no_of_documents = 12040167
=> 12040167
[4] pry(main)> batch_size = 10000
=> 10000
[5] pry(main)> throttle_delay = 1.minute
=> 1 minute
[6] pry(main)> (no_of_documents / batch_size) * throttle_delay / 1.hour
=> 20

Related to #425195 (closed)

Edited by Ravi Kumar

Merge request reports

Loading