Fetches into repos with many refs are slow
When fetching into repos with many refs, then the fetch can be extremely slow even if only fetching a small bunch of objects. The root cause is mostly that we're loading all referenced commits, so most of the time is spent unpacking objects.