Create one "CommitFinder" per Rails request
We have recently witnessed a Git performance degradation at a customer site where bringing back some of the pre-Gitaly-migration Rugged code into Unicorn resulted in better performance. I wonder if this is because we have some fixed overhead for starting
git cat-file, which we pay on each Gitaly call, while Rugged only has this overhead once per gitlab-rails request.
We could make a new bidirectional RPC "CommitFinder" that would act as a wrapper around
git cat-file. A message requesting a commit goes in, one commit goes back, repeat. At the end of the rails request a middleware would have to signal to Gitaly that the RPC is done.
This won't be as efficient as looking up multiple commits in a row with Rugged because it still incurs a network round trip for each commit we look up. But the overhead will be lower than calling FindCommit N times in a row within one Rails request.
In a way this means capitulating to the N+1 problem, and just making it more efficient to do N+1 queries. That seems sub-optimal but maybe it is really the more viable way to achieve and maintain good performance as GitLab's application code develops over time.
One thing we do need in the first iteration is a good story around monitoring. All our latency monitoring is currently based on RPC begin/end times. With the CommitFinder approach, that number will be useless because it includes an unknown number of commit lookups, plus idle time.