Skip to content

Eliminate Gitaly N+1 queries loading submodules

Stan Hu requested to merge sh-reduce-gitaly-calls-submodules into master

When a directory contains many submodules, we make a number of Gitaly N+1 queries:

  1. One TreeEntry RPC to look up each submodule
  2. One TreeEntry RPC to read the .gitmodules file

The first item is redundant because we've already determined the tree commit is a submodule. We can eliminate this query by skipping it altogether.

The second item can be optimized by caching the contents of .gitmodules once per directory. We already have a mechanism for caching this in SubmoduleLinks. Now we just need to use it.

This significantly reduces Gitaly RPCs and closes THREE issues with one stone:

  1. #35333 (closed)
  2. #194380 (closed)
  3. #38458 (closed)
Edited by Stan Hu

Merge request reports