Index an ancestor-aware active field and route the group tree filter through it

What does this MR do and why?

Makes the group tree and group listing filter searches Elasticsearch-eligible. Two halves.

1. An ancestor-aware active field on the group document. This is the blocker. Every one of these surfaces sends active on every request — vue_shared/components/groups_list/resolvers.js:7,10 and groups/show/graphql/resolvers.js:100,102 — and active was not allowlisted, so allowed_filters? was false and each request fell back to PostgreSQL.

active is not a plain flag. Group.active is self_and_ancestors_active = self_and_ancestors_non_archived.self_and_ancestors_not_aimed_for_deletion (app/models/group.rb:262-263), while the index stored only archived, the group's own flag. Indexing the ancestor-aware value is what makes the filter expressible, and it is the same gap that kept archived out of ALLOWED_ES_FILTERS.

  • Mapping and computed field on Types::Group / References::Group, SCHEMA_VERSION bumped
  • AddActiveToGroups adds the mapping, BackfillActiveOnGroups fills existing documents
  • by_active filter, registered in GroupQueryBuilder
  • expressible_active? declines while the backfill is unfinished, since documents written before it carry no active field and a term filter would silently drop them. Separate from index_ready?, so only requests passing active wait

2. Routing the filter param through the finder. GroupDescendantsFinder#descendant_groups goes through Namespaces::GroupsFinder behind namespaces_advanced_groups_finder, with the original query kept as the fallback. GroupTree passes search: safe_params[:filter] from the dashboard and explore controllers. search_include_parents: false preserves the own-path matching those callers relied on through Namespace.search's default.

Invalidation

An ancestor-aware field goes stale on descendants when an ancestor changes, so Search::Elastic::GroupActiveStateWorker reindexes self_and_descendants. Three triggers, because active? has two independent inputs:

Trigger Where
archive / unarchive EE::NamespaceSetting, if: :archived_previously_changed?
mark for deletion EE::Groups::MarkForDeletionService#post_success
restore EE::Groups::RestoreService#post_success

The archive hook sits on NamespaceSetting because archived lives there, so no Group callback ever fires for it. That also fixes a pre-existing staleness in the existing archived field.

preload_indexing_data resolves the active state for a whole batch in one query. Group#active? walks the hierarchy per record, so serializing without this was N+1 in the number of documents — caught by the existing N+1 spec.

References

Related to #333635

Zero-downtime reindexing needs no work here: ::Group is in ES_SEPARATE_CLASSES, so create_standalone_indices reads the new mapping at reindex time. See #611247 (closed).

No changelog entry: behind namespaces_advanced_groups_finder, default_enabled: false.

Expected impact

~66 database-hours per week once rolled out, measured over 7 days on searching requests only:

Endpoint Searching DB/week Addressable
Groups::ChildrenController#index 216 h ~65 h
Dashboard::GroupsController#index 3 h ~1 h
Explore::GroupsController#index 1 h ~0.3 h

The group tree is ~30% search-specific, unlike /groups/:id/subgroups at ~2/3, because it pays for hierarchy expansion and ancestor loading whether or not a filter is present.

Screenshots or screen recordings

No UI change. Both backends return the same rows in the same order, which is what the parity specs assert.

How to set up and validate locally

Requires Elasticsearch and a licence with advanced search.

  1. Run the migrations, then enable the flag and index a group you belong to:

    Elastic::DataMigrationService.migration_has_finished?(:backfill_active_on_groups) # => must be true
    Feature.enable(:namespaces_advanced_groups_finder)
    
    group = Group.find_by_full_path('flightjs')
    Elastic::ProcessInitialBookkeepingService.track!(group, *group.descendants)
  2. Confirm the param set the frontend sends is now eligible, where it previously was not:

    params = { parent: group, include_parent_descendants: true, all_available: true,
               active: true, search: 'flight', search_include_parents: false }
    Search::AdvancedFinders::GroupsFinder.new(user, params).use_elasticsearch_finder?
    # => true
  3. Open a group's Subgroups and projects tab and filter. Confirm which backend answered via the log — elasticsearch_calls is only written when a request called Elasticsearch, and sits next to db_duration_s:

    tail -f log/development_json.log | grep --line-buffered "ChildrenController"
  4. Archive a parent group, then confirm its descendants are reindexed and drop out of an active: true filter.

  5. Disable namespaces_advanced_groups_finder and repeat. Results and order should be identical.

Known gaps

  • Unbounded cascade. Archiving a large top-level group enqueues one job walking every descendant. each_batch and throttled urgency keep it civil, but it is a candidate for per-batch jobs if it proves heavy.
  • archived stays PostgreSQL-only. The new field makes it expressible, but allowlisting it is a separate change.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Siddharth Dungarwal

Merge request reports

Loading
Loading