Investigate optimization opportunities for FindAllTags
While taking a look at Continuous Profiler's graphs for Git, we noticed that FindAllTags
takes up quite an amount of CPU cycles. It's thus a worthwhile goal to explore whether it could be optimized.
Two proposed opportunities:
-
Improve the caching strategy, as proposed by @zj-gitlab. We currently invalidate the complete cache as soon as any tag gets pushed. One could optimize this by instead adjusting the cache based on the incoming change.
-
Improve the implementation. We're calling into
git for-each-ref
to find all tags, and for each tag found we feed it into a long-livinggit cat-file --batch
process. This is implemented such that we repeatedly cycle between these processes, taking each line from the first, writing it into the second and then await its output. This access pattern is quite inefficient. We could probably optimize it by using a Go pipeline with three steps: 1. Read fromgit for-each-ref
, 2. Write the reference intogit cat-file
, 3. Process the output ofgit cat-file
.
There surely are additional optimization opportunities.