Skip to content

Switch to roaring bitmaps for the loader dependency caches

Description

In order to support loading larger projects (on the order of 100k elements) in less than 30G of RAM the LoadElement dependency cache needed improvements.

This MR does that by replacing the dict based cache with a roaring bitmap based cache instead.

https://roaringbitmap.org/

With this change, I can bst show debian-stack.bst in about 5.5G of RAM on my Linux laptop.

Changes proposed in this merge request:

  • Introduce dependency on roaringbitmap
  • Use RoaringBitmap and ImmutableRoaringBitmap in LoadElement

Merge request reports

Loading