Use search service for project autocomplete
What does this MR do and why?
Autocomplete is slow for searching projects
and groups
. This is because these are doing global searches using database calls and on every call it has to compute current_user.authorized_projects
and current_user.authorized_groups
.
This MR changes the projects autocomplete from always using a database search to using the SearchService
which uses Elasticsearch if it's available which should be quicker.
The change is behind a feature flag: autocomplete_projects_use_search_service
Future
I'm thinking we can extend this to search over fewer fields (just project name, namespace name) which will be much faster and better aligned with autocomplete behaviour.
We can also cache heavy operations like determining a user's authorized projects and groups for a few minutes so that subsequent autocomplete and search calls are much faster. I think this will have the biggest impact on autocomplete slowness.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
How to set up and validate locally
- Disable the feature flag:
Feature.disable(:autocomplete_projects_use_search_service)
- Perform an autocomplete: type
s
or/
in gitlab. Note the results - Enable the feature flag:
Feature.enable(:autocomplete_projects_use_search_service)
- Perform the same autocompletes and note the same results are returned
Related to #472011 (closed)
Merge request reports
Activity
changed milestone to %17.6
assigned to @maddievn
added pipelinetier-1 label
- A deleted user
added backend feature flag labels
- Resolved by Madelein van Niekerk
Reviewer roulette
Category Reviewer Maintainer backend @hmerscher
(UTC+2, same timezone as author)
@lma-git
(UTC-7, 9 hours behind author)
Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger- Resolved by John Mason
- Resolved by Madelein van Niekerk
- Resolved by John Mason
Hi @rkumar555
can you please review this and send to@terrichu
once you're happy?
requested review from @rkumar555
@terrichu Could you please do the maintainer review?
requested review from @terrichu and removed review request for @rkumar555
added pipeline:mr-approved label
added pipelinetier-3 pipeline:run-e2e-omnibus-once labels and removed pipelinetier-1 label
Before you set this MR to auto-merge
This merge request will progress on pipeline tiers until it reaches the last tier: pipelinetier-3. We will trigger a new pipeline for each transition to a higher tier.
Before you set this MR to auto-merge, please check the following:
- You are the last maintainer of this merge request
- The latest pipeline for this merge request is pipelinetier-3 (You can find which tier it is in the pipeline name)
- This pipeline is recent enough (created in the last 8 hours)
If all the criteria above apply, please set auto-merge for this merge request.
See pipeline tiers and merging a merge request for more details.
E2E Test Result Summary
allure-report-publisher
generated test report!e2e-test-on-gdk:
test report for 81ff9cd9expand test summary
+------------------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------------+--------+--------+---------+-------+-------+--------+ | Create | 16 | 0 | 0 | 0 | 16 | ✅ | | Plan | 6 | 0 | 0 | 0 | 6 | ✅ | | Govern | 16 | 0 | 0 | 0 | 16 | ✅ | | Secure | 4 | 0 | 0 | 0 | 4 | ✅ | | Data Stores | 2 | 0 | 0 | 0 | 2 | ✅ | | Verify | 2 | 0 | 0 | 0 | 2 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+ | Total | 46 | 0 | 0 | 0 | 46 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+
e2e-test-on-cng:
test report for 81ff9cd9expand test summary
+------------------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------------+--------+--------+---------+-------+-------+--------+ | Verify | 50 | 0 | 15 | 10 | 65 | ✅ | | Monitor | 8 | 0 | 12 | 0 | 20 | ✅ | | Govern | 78 | 0 | 11 | 7 | 89 | ✅ | | Plan | 86 | 0 | 8 | 9 | 94 | ✅ | | Package | 24 | 0 | 14 | 0 | 38 | ✅ | | Create | 139 | 0 | 20 | 13 | 159 | ✅ | | Secure | 4 | 0 | 2 | 0 | 6 | ✅ | | Configure | 0 | 0 | 3 | 0 | 3 | ➖ | | Release | 5 | 0 | 1 | 0 | 6 | ✅ | | Data Stores | 33 | 0 | 10 | 0 | 43 | ✅ | | Manage | 1 | 0 | 9 | 0 | 10 | ✅ | | Ai-powered | 0 | 0 | 2 | 0 | 2 | ➖ | | Growth | 0 | 0 | 2 | 0 | 2 | ➖ | | Fulfillment | 2 | 0 | 7 | 0 | 9 | ✅ | | ModelOps | 0 | 0 | 1 | 0 | 1 | ➖ | | Analytics | 2 | 0 | 0 | 0 | 2 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+ | Total | 432 | 0 | 117 | 39 | 549 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+
e2e-test-on-omnibus:
test report for fd80dd5aexpand test summary
+------------------------------------------------------------------+ | suites summary | +-------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +-------------+--------+--------+---------+-------+-------+--------+ | Data Stores | 66 | 0 | 20 | 0 | 86 | ✅ | | Package | 50 | 0 | 26 | 0 | 76 | ✅ | | Verify | 100 | 0 | 30 | 0 | 130 | ✅ | | Create | 678 | 0 | 100 | 0 | 778 | ✅ | | Monitor | 16 | 0 | 24 | 0 | 40 | ✅ | | Govern | 158 | 0 | 16 | 0 | 174 | ✅ | | Plan | 164 | 0 | 16 | 0 | 180 | ✅ | | Release | 10 | 0 | 2 | 0 | 12 | ✅ | | Analytics | 4 | 0 | 0 | 0 | 4 | ✅ | | Growth | 0 | 0 | 4 | 0 | 4 | ➖ | | Secure | 8 | 0 | 4 | 2 | 12 | ✅ | | Manage | 4 | 0 | 16 | 0 | 20 | ✅ | | Fulfillment | 4 | 0 | 14 | 0 | 18 | ✅ | | ModelOps | 0 | 0 | 2 | 0 | 2 | ➖ | | Configure | 0 | 0 | 6 | 0 | 6 | ➖ | | Ai-powered | 0 | 0 | 4 | 0 | 4 | ➖ | +-------------+--------+--------+---------+-------+-------+--------+ | Total | 1262 | 0 | 284 | 2 | 1546 | ✅ | +-------------+--------+--------+---------+-------+-------+--------+
removed pipeline:run-e2e-omnibus-once label
- Resolved by Madelein van Niekerk
- Resolved by John Mason
We can also cache heavy operations like determining a user's authorized projects and groups for a few minutes so that subsequent autocomplete and search calls are much faster. I think this will have the biggest impact on autocomplete slowness.
thought (non-blocking): This would be a big improvement. Even advanced search still uses users authorized projects for search (see #499615)
- Resolved by Madelein van Niekerk
reset approvals from @rkumar555 by pushing to the branch
requested review from @johnmason
requested review from @terrichu
mentioned in issue #472011 (closed)
started a merge train
mentioned in commit e5891e43
added workflowstaging-canary label
added workflowcanary label and removed workflowstaging-canary label
added workflowstaging label and removed workflowcanary label
added workflowproduction label and removed workflowstaging label
added workflowpost-deploy-db-staging label and removed workflowproduction label
added backportskip label
added workflowpost-deploy-db-production label and removed workflowpost-deploy-db-staging label
added releasedcandidate label
mentioned in issue gitlab-org/quality/triage-reports#20953 (closed)
added releasedpublished label and removed releasedcandidate label
added devopsfoundations label and removed devopsdata stores label
mentioned in issue gitlab-org/quality/triage-reports#21522 (closed)
mentioned in issue gitlab-org/quality/triage-reports#22026