When serving a fetch, git upload-pack copies data from a git pack-objects stdout pipe to its stdout. This commit increases the size of the buffer used for that copying from 8192 to 65515, the maximum sideband-64k packet size.
Previously, this buffer was allocated on the stack. Because the new buffer size is nearly 64KB, we switch this to a heap allocation.
On GitLab.com we use GitLab's pack-objects cache which does writes of 65515 bytes. Because of the default 8KB buffer size, propagating these cache writes requires 8 pipe reads and 8 pipe writes from git-upload-pack, and 8 pipe reads from Gitaly (our Git RPC service). If we increase the size of the buffer to the maximum Git packet size, we need only 1 pipe read and 1 pipe write in git-upload-pack, and 1 pipe read in Gitaly to transfer the same amount of data. In benchmarks with a pure fetch and 100% cache hit rate workload we are seeing CPU utilization reductions of over 30%.
Signed-off-by: Jacob Vosmaer jacob@gitlab.com