Skip to content

Avoid N+1 queries on project search

About

When working on adding compliance framework settings on project view (!29137 (merged)), we've noticed some test failures on the N+1 tests around searching projects (https://gitlab.com/gitlab-org/gitlab/-/jobs/508606713)

@ifarkas has curiously raised some questions around why other missing preloads do not cause the test to fail

I was just curious seeing that in Dashboard::ProjectsController we also preload creator, or project_feature. Those aren't preloaded here, yet the spec doesn't fail because of N+1, but it failed for compliance_framework_setting. 🤷

!29137 (comment 329767783)

Investigation

  • The N+1 test seems to be incorrectly set up to capture these addition N+1 hotspots on creator and project_feature. I suspect that we are using the same creator and project_feature and cached queries are not counted.
  • I have tried to dig in the performance bar results in Gitlab.com search page and confirmed that there are quite a lot of N+1 queries around creator and project_feature

Proposal

  • Fix ./ee/spec/features/search/elastic/global_search_spec.rb test to correctly pick up N+1 queries
  • Add appropriate preloads to improve performance of search page

Notes

  • There could be other search scopes (e.g. issues, milestones) that could exhibit N+1 queries too
  • How we can avoid missing out on these new preloads moving forward
    • Can we consolidate these preloads? At the moment, we have these logic in two places
      • The controller (i.e Dashboard::ProjectsController)
      • The elastic module (i.e. Elastic::SearchResults
Edited by Tan Le