Backfill group milestones in Advanced Search

What does this MR do and why?

Adds the Advanced Search migration ReindexMilestones (ee/elastic/migrate/20261012155945_reindex_milestones.rb), with its docs yml and spec. It reindexes all milestones so group milestones created before group milestone indexing existed get documents.

It uses Search::Elastic::MigrationDatabaseBackfillHelper:

  • Batch size is 50,000, the same as ReindexAllMilestonesFromDatabase, with a 1-minute throttle delay and retry on failure.
  • It respects limited indexing. It keeps only records where maintaining_elasticsearch? is true. Group milestones pass through Milestone#use_elasticsearch?, which falls back to the group.
  • Under limited indexing, that check does one cached lookup per distinct project or group, not per milestone: ElasticsearchEnabledCache keeps each result for a day, and a cache miss costs one exists? query. This comes from the shared helper, and self-managed backfills like BackfillWorkItems, ReindexAllIssuesFromDatabase and BackfillGroupsToElasticsearch already run it at the same batch size.
  • It preloads the project namespace and group namespace settings, the same preload as the existing GitLab.com migration.

It reindexes all milestones instead of filtering project_id IS NULL. That filter scans past project milestones. The first batch read and discarded 181,131 rows to return 1,000 (about 88,400 buffers, 13,655 ms cold). The plain primary key range takes 215 buffers and 5.8 ms per batch. Plan: https://console.postgres.ai/shared/8ccf384b-d944-44a8-bc7c-737da0502e83

Project milestone documents are rewritten in place (same _id and routing), so nothing is duplicated.

The migration is skipped on GitLab.com (skip_if -> { saas_with_es? }). ReindexAllMilestonesFromDatabase (ee/elastic/migrate/20260929120002_reindex_all_milestones_from_database.rb, from !258497 (merged)) already reindexes every milestone there. It runs after !257456 (merged) is deployed, so it indexes group milestones too. That migration skips everything that isn't GitLab.com, so self-managed and Dedicated need this one.

Why this is needed when !258497 (merged) already reindexes milestones

All eight migrations in !258497 (merged), including the milestone one, have skip_if -> { !saas_with_es? }. They repair INC-14518, which only affected GitLab.com, so they never run on self-managed or Dedicated.

Instance Group milestones backfilled by !258497 (merged) Backfilled by this MR
GitLab.com Yes No (skipped)
Self-managed and Dedicated No Yes

Without this MR, those instances index new and updated group milestones after upgrading, but every group milestone created before the upgrade stays unindexed until it's edited or an admin runs a full reindex. Group milestone search would then return incomplete results once the advanced_search_group_milestones flag is removed.

The flag is gitlab_com_derisk, so the backfill could wait for the flag-removal MR. It ships now instead: it's harmless early, because !257456 (merged) keeps the documents in sync from here on, and it lets self-managed instances finish backfilling well before the flag is removed.

It is safe to run now. !257456 (merged) indexes group milestones and keeps their documents in sync on delete, transfer, archive, and limited-indexing changes. It is merged and deployed. Running a backfill before that would have left orphaned or duplicate documents.

Nothing becomes visible in search yet. Group milestone documents only show up once !260603 authorizes them in the query, behind a feature flag. The scope decision for that is in #632917. It doesn't affect this backfill, because the documents already carry namespace_id, visibility_level and traversal_ids.

The spec covers:

  • The migration is skipped on GitLab.com and not elsewhere.
  • The shared "migration reindexes all data" examples.
  • Group milestones are routed by their root ancestor (n_<root_id>).
  • No N+1 queries across project and nested-group milestones.
  • Under limited indexing, only milestones in indexed namespaces are tracked.

References

Closes #630857

Screenshots or screen recordings

Not applicable. This is a backend-only migration with no UI change.

How to set up and validate locally

Not applicable beyond the spec. The documents are not reachable from search until !260603 lands, so no manual step shows a difference.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

🤖 Generated with Claude Code

Edited by Ravi Kumar

Merge request reports

Loading
Loading