Remove hashed_root_namespace_id from users index documents
What does this MR do and why?
The field hashed_root_namespace_id has already been removed from the UserInstanceProxy but there are some documents that have this field. This migration will use the helper Elastic::MigrationRemoveFieldsHelper to remove this field from the documents.
Screenshots or screen recordings
Screenshots are required for UI changes, and strongly recommended for all other merge requests.
| Before | After |
|---|---|
How to set up and validate locally
-
Ensure elasticsearch is enabled
Prepare the data
-
Run the following commands in the rails console
client = ::Gitlab::Search::Client.new
client.indices.put_mapping(index: User.__elasticsearch__.index_name, body: { properties: { hashed_root_namespace_id: { type: 'integer' } } })
client.update_by_query(index: User.__elasticsearch__.index_name, refresh: true, body: { script: { source: "ctx._source.hashed_root_namespace_id=1" } })
- Now run the following command in rails console and ensure the count is greater than
0
client.count(index: User.__elasticsearch__.index_name, body: { query: { bool: { must: { exists: { field: 'hashed_root_namespace_id' } } } } })['count']
- Now run the migration by the following command in the rails console
Elastic::DataMigrationService[20230928104245].send(:migration).migrate
- Now ensure the count is
0by running the following command in console
client.count(index: User.__elasticsearch__.index_name, body: { query: { bool: { must: { exists: { field: 'hashed_root_namespace_id' } } } } })['count']
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Run time
~ 20 hours
[3] pry(main)> no_of_documents = 12040167
=> 12040167
[4] pry(main)> batch_size = 10000
=> 10000
[5] pry(main)> throttle_delay = 1.minute
=> 1 minute
[6] pry(main)> (no_of_documents / batch_size) * throttle_delay / 1.hour
=> 20
Related to #425195 (closed)
Edited by Ravi Kumar