Backport of 'Fix Zoekt indexing by cleaning up replicas without indices'
What does this MR do and why?
This MR backports !222058 (merged) which fixes a critical bug in Zoekt indexing for namespaces with missing indices.
Problem: The original fix (MR !221451 (merged)) successfully selected namespaces with missing indices, but indices were still not being created due to a deeper issue in the rollout flow. PlanningService only checked replica count and returned :unchanged for namespaces with correct replica count but 0 indices, causing ProvisioningService to skip them entirely.
Solution: Instead of trying to fix PlanningService's logic, this takes a simpler approach: delete all replicas without indices upfront. This forces a replica count mismatch, which triggers the existing PlanningService flow to recreate them properly with indices.
Customer Impact: Self-managed instances with Zoekt enabled experienced non-functional code search despite proper configuration. Namespaces would have replicas created but no indices, requiring manual intervention to trigger indexing.
Changes:
- Add
without_indicesscope to Replica model - Add cleanup in RolloutService that runs before SelectionService
- Use batched deletion (
each_batch) for efficiency and safety - Add comprehensive test coverage for cleanup functionality
Backport notes for 18.7:
- Cherry-picked cleanly with no conflicts
✅
Related:
- Original MR: !222058 (merged)
- Reverted MR: !221451 (merged)
- Original issue: #588267 (closed)
- Request for Help: https://gitlab.com/gitlab-com/request-for-help/-/issues/4126
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
- This MR is backporting a bug fix, documentation update, or spec fix, previously merged in the default branch.
- The MR that fixed the bug on the default branch has been deployed to GitLab.com (not applicable for documentation or spec changes).
- The MR title is descriptive (e.g. "Backport of 'title of default branch MR'"). This is important, since the title will be copied to the patch blog post.
-
Required labels have been applied to this merge request
- severity label and bug subtype labels (if applicable)
- If this MR fixes a bug that affects customers, the customer label has been applied.
- This MR has been approved by a maintainer (only one approval is required).
-
Ensure the
e2e:test-on-omnibus-eejob has succeeded, or if it has failed, investigate the failures. If you determine the failures are unrelated, you may proceed. If you need assistance investigating, reach out to a Software Engineer in Test in #s_developer_experience.
Note to the merge request author and maintainer
If you have questions about the patch release process, please:
- Refer to the patch release runbook for engineers and maintainers for guidance.
- Ask questions on the
#releasesSlack channel (internal only). - Once the backport has been merged, the commit changes will be automatically deployed to a release environment that can be used for manual validation. See after merging runbook for details.