Index group milestones and keep them in sync with group changes
What
Group milestones are indexed in Elasticsearch and kept in sync with their group. This ships in one MR, !257456 (merged), so no group milestone enters the index before the sync code exists.
Changes
- The group milestone Elasticsearch document includes the group's
namespace_id,traversal_ids,visibility_level, andarchived. - Group milestone documents are routed by
n_<root_ancestor_id>, based on the group's traversal ids. Project milestones keep theirproject_<id>routing. Milestone#use_elasticsearch?is overridden so project milestones useproject.use_elasticsearch?and group milestones usegroup.use_elasticsearch?. Before, group milestones fell back toproject&.use_elasticsearch?, which isnil, so they were never indexed.- Group milestone documents are removed when a group is deleted, transferred to a new root, or removed from limited indexing (
Search::ElasticGroupAssociationDeletionWorker). - A group's milestones are reindexed:
- when its visibility changes or the group is newly enabled for indexing:
Elastic::ProcessBookkeepingService.maintain_indexed_namespace_associations!now includesmilestones, the same way it already includesepics. - when it's transferred, along with its subgroups' milestones:
EE::Groups::TransferService. - when it's archived or unarchived, along with its subgroups' milestones:
Search::Elastic::GroupArchivedEventWorker.
- when its visibility changes or the group is newly enabled for indexing:
gitlab:elastic:index_group_entitiesindexes group milestones;gitlab:elastic:indexalready covers them throughElasticNamespaceIndexerWorker. Group import (direct transfer and file-based) indexes them through the normal create callback.
Group milestones created before this MR is deployed have no documents yet. Backfilling them is a separate issue, #630857, that ships once this MR is deployed.
Why indexing and sync must ship together
milestones.group_id is ON DELETE CASCADE, so a deleted group's milestone rows vanish without callbacks; if those milestones were already indexed, their documents would be orphaned forever. A group transfer changes its routing (n_<root_ancestor_id>), so the next milestone update would write a second copy under the new routing while the old copy stays behind on the old shard. Turning on indexing before this sync code exists would leave both cases unrecoverable.
Notes
#630618 still filters group milestone documents out of search results until it lands, so group milestones aren't searchable until then.
Part of #428589. Based on the implementation plan.