Skip to content

blob: Optimize Go implementation of LFS pointer lookup

LFS pointers are now live in production, but I've noticed that their 95% quantiles perform worse than the Ruby implementation. I've thus decided to take another look at it and do some optimizations. In this MR, I introduce a first iteration of a benchmarking framework by introducing a new bench target to our Makefile and a benchmarking repo, which is for now just gitlab-org/gitlab. That can and should be improved in future iterations.

With this benchmark, I was able to exercise some performance tweaks I had in mind. Which in total leads to the following performance improvements of the Go implementations:

        # Before
        BenchmarkFindLFSPointers/limitless-16                      1        42812541563 ns/op
        BenchmarkFindLFSPointers/limit-16                          1        14662540521 ns/op
        BenchmarkReadLFSPointers/limitless-16                      1        47841769495 ns/op
        BenchmarkReadLFSPointers/limit-16                   1000000000               0.362 ns/op
    
        # After
        BenchmarkFindLFSPointers/limitless-16                      1        20154639708 ns/op
        BenchmarkFindLFSPointers/limit-16                   1000000000               0.180 ns/op
        BenchmarkReadLFSPointers/limitless-16                      1        18473904116 ns/op
        BenchmarkReadLFSPointers/limit-16                   1000000000               0.108 ns/op

So a nice 2-3x speedup for the limitless RPCs, and a 1.000.000.000 speedup for the limited one. I cannot quite trust the 0.18ns/op, but results are consistent 🤷

Relates to #2954 (closed)

Edited by Mark Wood

Merge request reports

Loading