Add advanced finder and visibility level filter

What does this MR do and why?

Group listing and searching is currently always served by PostgreSQL through GroupsFinder. Searching by name uses a trigram text match, which is expensive on large datasets. Groups are already indexed in Elasticsearch (there is a groups index with name, path, description, visibility, parent and hierarchy fields), but nothing outside the global search page uses that index today.

This MR adds the machinery needed to serve group listing and searching from Elasticsearch, behind a feature flag that is off by default:

  • A new Search::AdvancedFinders::GroupsFinder (a CE stub plus the real EE implementation) queries the groups Elasticsearch index, following the same shape as the existing work items advanced finder.
  • A new Namespaces::GroupsFinder decides which backend serves a request, and falls back to GroupsFinder when Elasticsearch cannot handle the request.
  • Elasticsearch does the filtering, but PostgreSQL does the sorting. This is because the groups index stores name and path as analyzed text, which Elasticsearch cannot sort on, and the default sort for both the REST API and GraphQL is by name. Sorting the re-hydrated records in PostgreSQL keeps every sort option working and makes the ordering identical to the PostgreSQL path by construction, since it reuses the same sorting code.
  • The finder returns an ActiveRecord::Relation rather than an Elasticsearch relation object, so existing callers keep working unchanged, including pagination with Kaminari and further scope chaining.
  • The finder declines and falls back to PostgreSQL in two situations: when any requested filter cannot be expressed in Elasticsearch (for example min_access_level, owned, or with_statistics), and when the match set is larger than 1000 records, since sorting only part of a result set would return the wrong groups.
  • Two new reusable Elasticsearch filters are added, by_ids and by_visibility_level, wired into the group query builder.
  • No call sites are migrated in this MR, so there is no behavior change yet. Later MRs move GraphQL, the REST API, and autocomplete onto the new finder.
  • Every filter that is allowed to go to Elasticsearch has a spec comparing its results against GroupsFinder, so the two backends cannot silently diverge.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

  1. Enable Elasticsearch in the GDK and ensure groups are indexed.

  2. Enable the feature flag in the Rails console:

    Feature.enable(:advanced_groups_finder_elasticsearch)
  3. In the Rails console, compare the two finders directly, for example:

    Namespaces::GroupsFinder.new(User.first, search: 'test').execute
    GroupsFinder.new(User.first, search: 'test').execute

    and check that they return the same groups.

  4. Confirm it falls back to PostgreSQL for a filter Elasticsearch cannot express, for example by passing:

    Namespaces::GroupsFinder.new(User.first, min_access_level: Gitlab::Access::DEVELOPER).execute

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #333635

Edited by Siddharth Dungarwal

Merge request reports

Loading
Loading