Skip to content

git: Do not write commit graphs on fetches

Patrick Steinhardt requested to merge pks-fetch-dont-write-commit-graphs into master

In production, we're observing that FetchSourceBranch() is so slow in repos with many references that it's often exceeding its deadline and thus fails. To fix this, we have applied a custom patch via caf2cfab (git: Speed up fetches in repos with many refs, 2021-08-05) which speeds up loading of references in git-fetch(1). While this patch has been effective and sped up FetchSourceBranch by about 50%, we still regularly hit the deadline.

Logs of such failed RPC calls curiously show a "Collecting referenced commits" progress bar, indicating that we're in fact busy writing a commit-graph. This is quite surprising: Gitaly never passes the relevant option, and neither do we instruct git-fetch(1) via a switch. But as it turns out, both CNG and Omnibus have the config "fetch.writeCommitGraph" set in their gitconfig by default.

Digging back in history, this has been implemented by booking.com, e.g. in 0776d26fe (omnibus-gitconfig: Add writeCommitGraph gitconfig, 2020-03-31). The reasoning was to speed up mirrors, and that the option to write commit graphs has also been enabled by default in git-gc(1). But since then, Gitaly has grown the architecture to write commit graphs via its usual maintenance RPCs RepackFull(), RepackIncremental() and GarbageCollect(). Arguably, this is the correct place to update such auxiliary information: we don't want to pay this penalty on user-facing RPCs, but instead want to do it in background jobs which don't impact the user.

So let's regain control over writing commit-graphs in git-fetch(1) by unconditionally disabling this option. Ownership of the gitconfig in Gitaly distributions like CNG and Omnibus are deprecated anyway such that we have a single place which is responsible for it.

Part of gitlab#336657 (closed)

Merge request reports