Clear tracking queues when recreating index from scratch
What does this MR do and why?
I ran into this issue locally because my gdk had items queued for indexing (meaning they are queued in the redis ZSET used for tracking Elasticsearch records awaiting indexing). A recent change was made to the format of the data when it is in the queue.
I ran gitlab:elastic:index to recreate my index and kept getting NoMethodError and records were stuck in the queues
NoMethodError: undefined method
as_indexed_json' for an instance of Elastic::MultiVersionInstanceProxy from /Users/terrichu/Developer/gdk/gitlab/ee/lib/gitlab/elastic/document_reference.rb:126:inas_indexed_json'
I saw in the queues that work items were all duplicated but in two different formats (| is the new delimiter and is the legacy delimiter). The one with the delimiter was causing the no_method_error
"WorkItem 302 302 group_7", 25237.0
"WorkItem|302|group_7", 31922.0
This MR adds a step to a full indexing to clear the tracking for all of the bookkeeping queues.
AI Summary
This change adds a new cleanup step to the search indexing process. When the system rebuilds its search index (which helps users find content faster), it now also clears out old tracking queues that keep track of what needs to be indexed. This prevents leftover tasks from previous indexing runs from interfering with the new indexing process, making the search rebuild more reliable and clean.
References
Screenshots or screen recordings
| Before | After |
|---|---|
How to set up and validate locally
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.