git: Do not write commit graphs on fetches
In production, we're observing that FetchSourceBranch()
is so slow in
repos with many references that it's often exceeding its deadline and
thus fails. To fix this, we have applied a custom patch via caf2cfab
(git: Speed up fetches in repos with many refs, 2021-08-05) which speeds
up loading of references in git-fetch(1). While this patch has been
effective and sped up FetchSourceBranch by about 50%, we still regularly
hit the deadline.
Logs of such failed RPC calls curiously show a "Collecting referenced commits" progress bar, indicating that we're in fact busy writing a commit-graph. This is quite surprising: Gitaly never passes the relevant option, and neither do we instruct git-fetch(1) via a switch. But as it turns out, both CNG and Omnibus have the config "fetch.writeCommitGraph" set in their gitconfig by default.
Digging back in history, this has been implemented by booking.com, e.g.
in 0776d26fe (omnibus-gitconfig: Add writeCommitGraph gitconfig,
2020-03-31). The reasoning was to speed up mirrors, and that the option
to write commit graphs has also been enabled by default in git-gc(1).
But since then, Gitaly has grown the architecture to write commit graphs
via its usual maintenance RPCs RepackFull()
, RepackIncremental()
and
GarbageCollect()
. Arguably, this is the correct place to update such
auxiliary information: we don't want to pay this penalty on user-facing
RPCs, but instead want to do it in background jobs which don't impact
the user.
So let's regain control over writing commit-graphs in git-fetch(1) by unconditionally disabling this option. Ownership of the gitconfig in Gitaly distributions like CNG and Omnibus are deprecated anyway such that we have a single place which is responsible for it.
Part of gitlab#336657 (closed)