Skip to content
Snippets Groups Projects

Use search service for project autocomplete

All threads resolved!

What does this MR do and why?

Autocomplete is slow for searching projects and groups. This is because these are doing global searches using database calls and on every call it has to compute current_user.authorized_projects and current_user.authorized_groups.

This MR changes the projects autocomplete from always using a database search to using the SearchService which uses Elasticsearch if it's available which should be quicker.

The change is behind a feature flag: autocomplete_projects_use_search_service

Future

I'm thinking we can extend this to search over fewer fields (just project name, namespace name) which will be much faster and better aligned with autocomplete behaviour.

We can also cache heavy operations like determining a user's authorized projects and groups for a few minutes so that subsequent autocomplete and search calls are much faster. I think this will have the biggest impact on autocomplete slowness.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

  1. Disable the feature flag: Feature.disable(:autocomplete_projects_use_search_service)
  2. Perform an autocomplete: type s or / in gitlab. Note the results
  3. Enable the feature flag: Feature.enable(:autocomplete_projects_use_search_service)
  4. Perform the same autocompletes and note the same results are returned

Related to #472011 (closed)

Edited by Madelein van Niekerk

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Reviewer roulette

    Category Reviewer Maintainer
    backend @hmerscher profile link current availability (UTC+2, same timezone as author) @lma-git profile link current availability (UTC-7, 9 hours behind author)

    Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • Madelein van Niekerk
  • Madelein van Niekerk changed the description

    changed the description

  • added 1 commit

    • fd80dd5a - Apply 2 suggestion(s) to 1 file(s)

    Compare with previous version

  • Madelein van Niekerk marked this merge request as ready

    marked this merge request as ready

  • requested review from @rkumar555

  • Ravi Kumar approved this merge request

    approved this merge request

  • @terrichu Could you please do the maintainer review?

  • Ravi Kumar requested review from @terrichu and removed review request for @rkumar555

    requested review from @terrichu and removed review request for @rkumar555

  • Before you set this MR to auto-merge

    This merge request will progress on pipeline tiers until it reaches the last tier: pipelinetier-3. We will trigger a new pipeline for each transition to a higher tier.

    Before you set this MR to auto-merge, please check the following:

    • You are the last maintainer of this merge request
    • The latest pipeline for this merge request is pipelinetier-3 (You can find which tier it is in the pipeline name)
    • This pipeline is recent enough (created in the last 8 hours)

    If all the criteria above apply, please set auto-merge for this merge request.

    See pipeline tiers and merging a merge request for more details.

  • E2E Test Result Summary

    allure-report-publisher generated test report!

    e2e-test-on-gdk: :white_check_mark: test report for 81ff9cd9

    expand test summary
    +------------------------------------------------------------------+
    |                          suites summary                          |
    +-------------+--------+--------+---------+-------+-------+--------+
    |             | passed | failed | skipped | flaky | total | result |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Create      | 16     | 0      | 0       | 0     | 16    | ✅     |
    | Plan        | 6      | 0      | 0       | 0     | 6     | ✅     |
    | Govern      | 16     | 0      | 0       | 0     | 16    | ✅     |
    | Secure      | 4      | 0      | 0       | 0     | 4     | ✅     |
    | Data Stores | 2      | 0      | 0       | 0     | 2     | ✅     |
    | Verify      | 2      | 0      | 0       | 0     | 2     | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Total       | 46     | 0      | 0       | 0     | 46    | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+

    e2e-test-on-cng: :white_check_mark: test report for 81ff9cd9

    expand test summary
    +------------------------------------------------------------------+
    |                          suites summary                          |
    +-------------+--------+--------+---------+-------+-------+--------+
    |             | passed | failed | skipped | flaky | total | result |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Verify      | 50     | 0      | 15      | 10    | 65    | ✅     |
    | Monitor     | 8      | 0      | 12      | 0     | 20    | ✅     |
    | Govern      | 78     | 0      | 11      | 7     | 89    | ✅     |
    | Plan        | 86     | 0      | 8       | 9     | 94    | ✅     |
    | Package     | 24     | 0      | 14      | 0     | 38    | ✅     |
    | Create      | 139    | 0      | 20      | 13    | 159   | ✅     |
    | Secure      | 4      | 0      | 2       | 0     | 6     | ✅     |
    | Configure   | 0      | 0      | 3       | 0     | 3     | ➖     |
    | Release     | 5      | 0      | 1       | 0     | 6     | ✅     |
    | Data Stores | 33     | 0      | 10      | 0     | 43    | ✅     |
    | Manage      | 1      | 0      | 9       | 0     | 10    | ✅     |
    | Ai-powered  | 0      | 0      | 2       | 0     | 2     | ➖     |
    | Growth      | 0      | 0      | 2       | 0     | 2     | ➖     |
    | Fulfillment | 2      | 0      | 7       | 0     | 9     | ✅     |
    | ModelOps    | 0      | 0      | 1       | 0     | 1     | ➖     |
    | Analytics   | 2      | 0      | 0       | 0     | 2     | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Total       | 432    | 0      | 117     | 39    | 549   | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+

    e2e-test-on-omnibus: :white_check_mark: test report for fd80dd5a

    expand test summary
    +------------------------------------------------------------------+
    |                          suites summary                          |
    +-------------+--------+--------+---------+-------+-------+--------+
    |             | passed | failed | skipped | flaky | total | result |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Data Stores | 66     | 0      | 20      | 0     | 86    | ✅     |
    | Package     | 50     | 0      | 26      | 0     | 76    | ✅     |
    | Verify      | 100    | 0      | 30      | 0     | 130   | ✅     |
    | Create      | 678    | 0      | 100     | 0     | 778   | ✅     |
    | Monitor     | 16     | 0      | 24      | 0     | 40    | ✅     |
    | Govern      | 158    | 0      | 16      | 0     | 174   | ✅     |
    | Plan        | 164    | 0      | 16      | 0     | 180   | ✅     |
    | Release     | 10     | 0      | 2       | 0     | 12    | ✅     |
    | Analytics   | 4      | 0      | 0       | 0     | 4     | ✅     |
    | Growth      | 0      | 0      | 4       | 0     | 4     | ➖     |
    | Secure      | 8      | 0      | 4       | 2     | 12    | ✅     |
    | Manage      | 4      | 0      | 16      | 0     | 20    | ✅     |
    | Fulfillment | 4      | 0      | 14      | 0     | 18    | ✅     |
    | ModelOps    | 0      | 0      | 2       | 0     | 2     | ➖     |
    | Configure   | 0      | 0      | 6       | 0     | 6     | ➖     |
    | Ai-powered  | 0      | 0      | 4       | 0     | 4     | ➖     |
    +-------------+--------+--------+---------+-------+-------+--------+
    | Total       | 1262   | 0      | 284     | 2     | 1546  | ✅     |
    +-------------+--------+--------+---------+-------+-------+--------+
  • Terri Chu
    • Resolved by John Mason

      We can also cache heavy operations like determining a user's authorized projects and groups for a few minutes so that subsequent autocomplete and search calls are much faster. I think this will have the biggest impact on autocomplete slowness.

      thought (non-blocking): This would be a big improvement. Even advanced search still uses users authorized projects for search (see #499615)

  • Terri Chu
  • Terri Chu requested changes

    requested changes

  • added 1 commit

    Compare with previous version

  • Madelein van Niekerk reset approvals from @rkumar555 by pushing to the branch

    reset approvals from @rkumar555 by pushing to the branch

  • requested review from @johnmason

  • requested review from @terrichu

  • John Mason approved this merge request

    approved this merge request

  • John Mason enabled automatic add to merge train when checks pass

    enabled automatic add to merge train when checks pass

  • John Mason resolved all threads

    resolved all threads

  • Madelein van Niekerk bypassed reviews on this merge request

    bypassed reviews on this merge request

  • merged

  • John Mason mentioned in commit e5891e43

    mentioned in commit e5891e43

  • added workflowstaging label and removed workflowcanary label

  • Please register or sign in to reply
    Loading