Skip to content

Advanced user search

Madelein van Niekerk requested to merge 377587-advanced-user-search into master

What does this MR do and why?

Implements advanced search for users. Currently user search performs a basic (database) search but this has performance issues and doesn't leverage the capabilities of elasticsearch.

Before this MR

  1. A user index has been created
  2. A feature flag to perform a callback on user and update elasticsearch was shipped
    • On gitlab.com it was fully enabled on 15 November
  3. A backfill migration was shipped
    • On gitlab.com the migration was completed on 20 November
  4. The first part of advanced search is to be shipped

This MR

Adds an ops feature toggle to switch from basic to advanced search for user searches. It also relies on two elastic migrations to be completed: create_user_index and backfill_users. (These are completed on gitlab.com).

This applies to web and API searches for all levels (global, group, project).

The advanced search has parity with its basic search counterpart which makes use of the UsersFinder:

  • Fuzzy match on name, username, email and public_email
  • Permissions
    • If the user does not have the ability to read a list of users, they get an empty list
    • If the user is not an admin:
      • They cannot see users in forbidden states
      • They cannot see users who match the search query on email, only public email
  • Level-based filters
    • For project level, only show users who are members of the project
    • For group level, only show users who are members of:
      • The group or projects of the groups
      • The group's ancestors or projects of the ancestors
      • The group's subgroups or projects of the subgroups
      • The same as above for shared groups
      • * this logic is same as before and moved to a shared location in !104392 (merged)

The elasticsearch query is built in UserClassProxy and doesn't use a basic_query_hash because we need to do a fuzzy search to match the basic search.

After this MR

  1. User searches will continue to be basic
  2. The feature flag to enable advanced user search has to be enabled. It is an ops flag with actors as users.
  3. Cleanup:

Screenshots or screen recordings

User_search_demo_local

How to set up and validate locally

  1. See how basic user search works by disabling the feature flag: Feature.disable(:advanced_user_search)
    1. Do a web global user search: http://127.0.0.1:3000/search?scope=users&search=admin
    2. Do a web project user search: http://127.0.0.1:3000/search?scope=users&search=admin&project_id=some_project_id
    3. Do a web group user search: http://127.0.0.1:3000/search?scope=users&search=admin&group_id=some_group_id
    4. Do the same for API searches
  2. Enable advanced user search by enabling the feature flag: Feature.enable(:advanced_user_search)
  3. Make sure elastic migrations are completed by running Elastic::MigrationWorker.new.perform, ElasticIndexInitialBulkCronWorker.new.perform and ElasticIndexBulkCronWorker.new.perform on repeat
  4. Perform advanced user searches:
    1. Do a web global user search: http://127.0.0.1:3000/search?scope=users&search=admin
    2. Do a web project user search: http://127.0.0.1:3000/search?scope=users&search=admin&project_id=some_project_id
    3. Do a web group user search: http://127.0.0.1:3000/search?scope=users&search=admin&group_id=some_group_id
    4. Do the same for API searches

Related to #377587 (closed)

Edited by Madelein van Niekerk

Merge request reports