The performance of the search results page is not great. It doesn't seem to matter whether it's basic search or advanced search with Elasticsearch. Below are several different profiles, including one with basic search and advanced search.
It seems that most of the slowness in search is due to rendering, not actually retrieving data.
Steps to reproduce
With or without advanced search enabled, search for any term that will likely come back with a fair number of results. Run a Requests Profile on the search.
Profiles
These profiles are quite large and it's hard to do them justice in a screenshot. I've taken a small screenshot of each and also provided a Dropbox link to download the entire profile. I wonder if both cases can be improved at the same time, but if further investigation reveals that one or each require more work then we can split into two issues.
With advanced search enabled (from customer):
Here there seem to be multiple problems - of course the show view is slow, just like with basic search. However, tons of 1-2% CI status lookups add considerable time, too.
@dblessing The Dropbox links didn't work for me. It's important to have the full profiles because the screenshots don't really capture the details here.
One issue appears to be an N+1 Gitaly query in the project results page: for 22 projects, there are 60 Gitaly find_commit requests to load the CI pipeline status:
From the profiles, it looks like there are 111 of these requests:
Ok, if you tweak the HTML profile threshold to 0.1, you can see that we start by trying to cache the pipeline statuses in Ci::ProjectPipelineStatus#load_in_batch_for_projects even though we cache the pipeline status, we don't cache the Gitaly commit result:
I also have the details in the description of #40260 (moved) which includes future plans on reducing this further.
Also, I've done another optimisation in !23952 (merged) which affects all search results pages that show the "Showing 1 - 5 of 5 ..." header. Although the projects search page doesn't seem to be one of these.
@stanhu Thanks for the two MRs addressing some components here. Will that close this out, or should we add this to a milestone and have the Editor Group do some further work? As an FYI we are bringing this one in to %12.2: https://gitlab.com/gitlab-org/gitlab-ee/issues/11777
@phikai I doubt that's it so far. There may be significant overlap with the Gitaly N+1 issues, though. I think we need to benchmark an instance with all the latest changes and break this issue into smaller issues.
@meks This issue is not scheduled for any milestone at this time. I'm not sure directly addressing this issue is actually possible. If you look at the history it already has 4 merge requests that addressed components of this, but the issue pretty vague. We do have some other performance issues open that we'd like to address as capacity permits.