Too many memory allocations in catfile.Batch methods
Here is a 30-second heap allocation profile captured recently on file-43, a Gitaly server that currently serves a lot of FindAllTags and FindLocalBranches traffic.
Note that the total amount is 12GB which is a lot. The biggest chunk (6.3GB) is in PostUploadPack, which we are already working on in gitlab-com/gl-infra&463 (closed). But what I found interesting is that there is so much other excessive heap allocation going on and it mostly has to do with catfile.Batch.
- 1.4GB of allocations in opentracing (because we create a span for each object we look up?)
- 2GB of allocations in parsing commits and tags (where we constantly create new
bufio
buffered readers)
Both of these things are fixable: the spans should probably not be created at this granularity, and parsing buffers can be reused.
Edited by Jacob Vosmaer