GitLab cache is masking Gitaly N+1 problems
As noted in https://gitlab.com/gitlab-org/gitlab-ce/issues/40260#note_47969375 and https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3356#note_47980287
Gitaly includes an N+1 detector which raises an exception if it detects that it is being called in a tight loop. The idea is to ensure that data is requested in batches, rather than individually, which is a good idea.
However, certain Gitaly calls will not happen if the data is already cached in Rails or RequestStore. This breaks the N+1 accounting, and can cause random-seeming failures - you visit a page which calls Gitaly in a loop, the first N items are cached, then the N+1 detector kicks in and raises an exception. If you visit the page again and again, eventually the items from Gitaly will be fully cached and the page will load without triggering the N+1 detector.
When the cache is cleared, e.g., on a deploy or during a primary -> secondary Geo failover, the N+1 detector will start tripping again.
I think we should modify the N+1 detector so it comes before values are (attempted to be) read from the cache. This will allow us to detect N+1 sites more reliably, and will prevent people from using caching as a hack to avoid fixing the underlying issue.