Enable Advanced User search regardless of namespace and license
Problem to solve
As a user of gitlab.com, I want to use advanced search for all user searches regardless of my namespace tier and whether or not the namespace (group/project) I am searching from is indexed into Elasticsearch so that I have a better and more performant experience. Secondary goals are to reduce violations in the Global Search team's error budget and load on the database that results from basic user searches.
Proposal
In elasticsearchable.rb
, override use_elasticsearch?
when the scope is 'users' so that it returns true when:
- The
advanced_user_search
feature flag is enabled. - The
create_user_index
andbackfill_users
migrations have completed. - The migration to reindex the user index has been completed: implemented in Elastic migration to reindex user index (!110728 - merged) or Draft: Recreate user index and backfill users (!110939 - closed).
- TBD: The columns
elasticsearch_search
andelasticsearch_indexing
exist in ApplicationSettings. https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/ee/application_setting.rb#L361
Data to back it up
Over the last 7 days
Non-elastic requests over 5 seconds have primarily come from SearchController#count
:
![](/-/project/278964/uploads/5c36638d608261f9db26523a00fdd2da/Screenshot_2023-02-03_at_09.25.05.png)
For the SearchController#count
endpoint, most of the requests over 5s are coming from users
and merge_requests
:
![](/-/project/278964/uploads/52fa97aefede80548e448ae4b38f1f75/Screenshot_2023-02-03_at_09.26.51.png)
Out of all non-elastic requests over 5 seconds, the vast majority is coming from searches with namespaces (group or project):
![](/-/project/278964/uploads/56568e5a3e104fa33d98c860ada01ad7/Screenshot_2023-02-03_at_09.27.42.png)
We don't have comprehensive data on which tier is used but we can see that the free
tier is called much more than paid tiers but we are missing majority of the data so can't draw a conclusion from that:
![](/-/project/278964/uploads/ebd1852224f5ce200be8ad8254b7a533/Screenshot_2023-02-03_at_09.33.57.png)
Our hypothesis is that if we enable advanced user search for all users, we will decrease the number of requests that take longer than 5 seconds by reducing the number of namespace calls which frequently violate the SLI.
The next step is to evaluate what we can do for Merge Requests.