Skip to content

localrepo: Speed up connectivity check when fetching into pooled repos

Patrick Steinhardt requested to merge pks-fetch-internal-objects-speedup into master

In production, we have observed that fetches into biggish pooled repos have started to time out. The root cause for this is most likely the connectivity check: when all objects have been retrieved, Git will verify that we have no missing objects. This connectivity check is implemented via git rev-list --not --all --alternate-refs --objects, where all changed references are written into the stdin of the command. While the command is already expensive by itself, it gets even more expensive because it also includes references from alternates. Given that object pools duplicate all publicly visible references from their originial pool member and then add several references on top e.g. for dangling refs, this is growing really expensive.

The following benchmark demonstrates this in gitlab-org/gitlab with parameters taken from a real request that has been canceled in production:

Benchmark #1: git fetch --no-tags target.git cbdb147e293828c4fa2bfc931679d5e2b070053d
  Time (mean ± σ):     28.524 s ±  0.501 s    [User: 27.219 s, System: 1.983 s]
  Range (min … max):   28.170 s … 28.878 s    2 runs

Benchmark #2: git -c core.alternateRefsCommand="# exit 0" fetch --no-tags target.git cbdb147e293828c4fa2bfc931679d5e2b070053d
  Time (mean ± σ):     16.390 s ±  0.029 s    [User: 15.451 s, System: 0.938 s]
  Range (min … max):   16.369 s … 16.411 s    2 runs

Summary
  'git -c core.alternateRefsCommand="# exit 0" fetch --no-tags target.git cbdb147e293828c4fa2bfc931679d5e2b070053d' ran
    1.74 ± 0.03 times faster than 'git fetch --no-tags target.git cbdb147e293828c4fa2bfc931679d5e2b070053d'

As can be seen, stubbing out alternate references provides a nice speedup of 1.74x.

Disable alternate refs for internal fetches. This really only serves as an initial testbed so we can easily feature-flag the option. If it proves to not surface any problems while providing above speedups, we can globally enable the option for git-fetch(1).

Changelog: performance

Closes: #3231 (closed)

Edited by Sami Hiltunen

Merge request reports