Index an ancestor-aware active field and route the group tree filter through it
What does this MR do and why?
Makes the group tree and group listing filter searches Elasticsearch-eligible. Two halves.
1. An ancestor-aware active field on the group document. This is the blocker. Every one of these surfaces sends active on every request — vue_shared/components/groups_list/resolvers.js:7,10 and groups/show/graphql/resolvers.js:100,102 — and active was not allowlisted, so allowed_filters? was false and each request fell back to PostgreSQL.
active is not a plain flag. Group.active is self_and_ancestors_active = self_and_ancestors_non_archived.self_and_ancestors_not_aimed_for_deletion (app/models/group.rb:262-263), while the index stored only archived, the group's own flag. Indexing the ancestor-aware value is what makes the filter expressible, and it is the same gap that kept archived out of ALLOWED_ES_FILTERS.
- Mapping and computed field on
Types::Group/References::Group,SCHEMA_VERSIONbumped AddActiveToGroupsadds the mapping,BackfillActiveOnGroupsfills existing documentsby_activefilter, registered inGroupQueryBuilderexpressible_active?declines while the backfill is unfinished, since documents written before it carry noactivefield and a term filter would silently drop them. Separate fromindex_ready?, so only requests passingactivewait
2. Routing the filter param through the finder. GroupDescendantsFinder#descendant_groups goes through Namespaces::GroupsFinder behind namespaces_advanced_groups_finder, with the original query kept as the fallback. GroupTree passes search: safe_params[:filter] from the dashboard and explore controllers. search_include_parents: false preserves the own-path matching those callers relied on through Namespace.search's default.
Invalidation
An ancestor-aware field goes stale on descendants when an ancestor changes, so Search::Elastic::GroupActiveStateWorker reindexes self_and_descendants. Three triggers, because active? has two independent inputs:
| Trigger | Where |
|---|---|
| archive / unarchive | EE::NamespaceSetting, if: :archived_previously_changed? |
| mark for deletion | EE::Groups::MarkForDeletionService#post_success |
| restore | EE::Groups::RestoreService#post_success |
The archive hook sits on NamespaceSetting because archived lives there, so no Group callback ever fires for it. That also fixes a pre-existing staleness in the existing archived field.
preload_indexing_data resolves the active state for a whole batch in one query. Group#active? walks the hierarchy per record, so serializing without this was N+1 in the number of documents — caught by the existing N+1 spec.
References
Related to #333635
Zero-downtime reindexing needs no work here: ::Group is in ES_SEPARATE_CLASSES, so create_standalone_indices reads the new mapping at reindex time. See #611247 (closed).
No changelog entry: behind namespaces_advanced_groups_finder, default_enabled: false.
Expected impact
~66 database-hours per week once rolled out, measured over 7 days on searching requests only:
| Endpoint | Searching DB/week | Addressable |
|---|---|---|
Groups::ChildrenController#index |
216 h | ~65 h |
Dashboard::GroupsController#index |
3 h | ~1 h |
Explore::GroupsController#index |
1 h | ~0.3 h |
The group tree is ~30% search-specific, unlike /groups/:id/subgroups at ~2/3, because it pays for hierarchy expansion and ancestor loading whether or not a filter is present.
Screenshots or screen recordings
No UI change. Both backends return the same rows in the same order, which is what the parity specs assert.
How to set up and validate locally
Requires Elasticsearch and a licence with advanced search.
-
Run the migrations, then enable the flag and index a group you belong to:
Elastic::DataMigrationService.migration_has_finished?(:backfill_active_on_groups) # => must be true Feature.enable(:namespaces_advanced_groups_finder) group = Group.find_by_full_path('flightjs') Elastic::ProcessInitialBookkeepingService.track!(group, *group.descendants) -
Confirm the param set the frontend sends is now eligible, where it previously was not:
params = { parent: group, include_parent_descendants: true, all_available: true, active: true, search: 'flight', search_include_parents: false } Search::AdvancedFinders::GroupsFinder.new(user, params).use_elasticsearch_finder? # => true -
Open a group's Subgroups and projects tab and filter. Confirm which backend answered via the log —
elasticsearch_callsis only written when a request called Elasticsearch, and sits next todb_duration_s:tail -f log/development_json.log | grep --line-buffered "ChildrenController" -
Archive a parent group, then confirm its descendants are reindexed and drop out of an
active: truefilter. -
Disable
namespaces_advanced_groups_finderand repeat. Results and order should be identical.
Known gaps
- Unbounded cascade. Archiving a large top-level group enqueues one job walking every descendant.
each_batchand throttled urgency keep it civil, but it is a candidate for per-batch jobs if it proves heavy. archivedstays PostgreSQL-only. The new field makes it expressible, but allowlisting it is a separate change.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.