Gitlab::GitalyClient::TooManyInvocationsError at /dashboard/projects
I got the following error when trying to load /dashboard/projects. I updated my GDK yesterday which worked fine. This morning the following happened. Reloading the page a few times fixed it.
cc @andrewn
Gitlab::GitalyClient::TooManyInvocationsError at /dashboard/projects
GitalyClient#migrate called 31 times from single request. Potential n+1?
The following call site called into Gitaly 7 times:
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/gitaly_client.rb:163:in `migrate'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/git/repository.rb:1193:in `gitaly_migrate'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/git/commit.rb:63:in `find'
/home/axil/git/gitlab/gdk/ce/gitlab/app/models/repository.rb:1077:in `find_commit'
/home/axil/git/gitlab/gdk/ce/gitlab/app/models/repository.rb:109:in `commit'
/home/axil/git/gitlab/gdk/ce/gitlab/app/models/project.rb:552:in `commit'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/cache/ci/project_pipeline_status.rb:10:in `commit'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/cache/ci/project_pipeline_status.rb:83:in `load_from_project'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/cache/ci/project_pipeline_status.rb:75:in `load_status'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/cache/ci/project_pipeline_status.rb:21:in `block in load_in_batch_for_projects'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/cache/ci/project_pipeline_status.rb:19:in `each'
/home/axil/git/gitlab/gdk/ce/gitlab/lib/gitlab/cache/ci/project_pipeline_status.rb:19:in `load_in_batch_for_projects'
/home/axil/git/gitlab/gdk/ce/gitlab/app/helpers/projects_helper.rb:228:in `load_pipeline_status'
/home/axil/git/gitlab/gdk/ce/gitlab/app/views/shared/projects/_list.html.haml:10:in `_app_views_shared_projects__list_html_haml___1214348951835595048_178051760'Current Behavior after !23307 (closed)
To get the pipeline status of a project at most 3 Gitaly calls are made:
-
repo.exists?(Check if repo exists before finding commit) -
repo.find_commit(Find HEAD of default branch) -
repo.default_branch(Used in saving to cache, to cache pipeline statuses of the default branch only)
The dashboard/projects page and other pages like projects/explore can show 20 projects per page. Making 3 calls per project puts us over the 35 call limit.
Proposed Solution
A project_repositories table is already planned and can be used in exists? and default_branch so that a DB lookup is made instead of a Gitaly call.
This lowers the number of calls to 1 per project and puts us under the limit.
Relevant MRs
Edited by Takuya Noguchi