Skip to content
  • Daniel Martí's avatar
    db: switch from goleveldb to badger/v2 · 1fa0dc21
    Daniel Martí authored
    Move the tree and scrutinizer packages to this newer database. It's
    maintained, well tested/benchmarked, and it's starting to be used more
    in projects like ChainSafe.
    
    One bonus is that it has extra features that might be useful in the
    future, like automatic compression or encryption.
    
    Below are the numbers from the Census benchmark, run as follows:
    
    	go test -bench=Census -benchtime=1x -cpu 1 -count 10 . -censusSize 1000 -onlyCreateCensus
    
    	name    old time/op    new time/op    delta
    	Census     3.44s ± 1%     3.45s ± 1%     ~     (p=0.579 n=10+10)
    
    	name    old alloc/op   new alloc/op   delta
    	Census    62.4MB ± 2%    85.7MB ± 0%  +37.31%  (p=0.000 n=9+10)
    
    	name    old allocs/op  new allocs/op  delta
    	Census     1.10M ± 2%     1.39M ± 1%  +26.34%  (p=0.000 n=10+10)
    
    Performance is unaffected (we're cpu-bound by math work). Memory usage
    goes up a bit, but this is mainly because badger pre-allocates quite a
    bit of memory when starting up, so that it can perform better. We reduce
    this factor by 4x via an option.
    
    The added code contains two copied files from ChainSafe/gossamer's
    lib/database (interface.go and table.go), and our own implementation on
    top of badger/v2. ChainSafe's implementation on top of badger is a bit
    wasteful, and resulted in about 5% performance loss. This way, we also
    don't need to pull in the entire gossamer module.
    
    The only remaining use of goleveldb is ethereum, both via its tm-db and
    ethereum modules. Leaving those alone for now, as their interface isn't
    used everywhere, and it's quite heavy.
    1fa0dc21