Skip to content

Use trieNode auth for notes Elasticsearch query

Terri Churequested to merge
tchu-use-trie-node-auth-in-search-for-notes into master

What does this MR do and why?

This change introduces a new feature flag called search_notes_use_membership_filter that modifies how note search authorization works in Elasticsearch. Instead of using the existing project IDs filter method, when enabled, the code will use a membership-based filtering approach for notes search, but only if a specific data migration has completed. The change includes:

  1. Adding a new feature flag configuration file
  2. Creating a new get_authorization_filter method that chooses between the old and new filtering approaches
  3. Updating tests to work with both filtering methods by using shared examples
  4. Making sure tests properly handle the feature flag being on or off

The change aims to improve search functionality for notes while ensuring backward compatibility. The implementation is designed to be safely rolled out with the feature flag initially disabled by default.

References

This MR is step 3 in improving notes query performance. Notes uses the legacy authorization in Elasticsearch queries which can send 1000s of project_id to Elasticsearch. This will improve performance for global and group searches (the same was previously done for code, merge requests, issues, etc). The query needs to be improved because it is now used during issue search. The plan for improving the query is:

  1. add traversal_ids to notes index
  2. backfill traversal_ids in notes index
  3. switch notes index to use new authorization in queries (behind a FF) (this MR)
  4. remove FF

Screenshots or screen recordings

Before After

How to set up and validate locally

testing this manually for each user type (anonymous, admin, logged in) and role is kind of a pain. Notes search is supported on 4 entities: merge requests, issues, snippets (project), and commits.

Due to this, I rely upon the visibility specs which test for global searches, group searches, and project searches that notes search is properly gated for each user type, role, and feature access level combination.

if you want to spot test any combo, you can setup data and perform searches to check

  1. enable elasticsearch in gdk
  2. enable advanced search in the admin - search page and index the instance
  3. create groups and projects at whatever visibility level you want to check
  4. set feature access level (this is done in project settings) for the note type
  5. create a user and invite them to the project or group
  6. add some notes (as an admin user or other user)
  7. perform global, group, and project searches to ensure you can see notes when you are supposed to see them

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Terri Chu

Merge request reports