Skip to content

skip unneeded files in managedBuildChunkHeap by sorting by health

David Vorick requested to merge sort-file-entries into master

This resolves a performance bottleneck when building chunks to add to the heap where the renter may look through thousands of files at once to find hundreds of bad chunks. Instead of looking through all of the files like that, we can sort by the cached health of the files and then only look at the N worst files if we only need N chunks.

This MR implements that, and takes the stalling that Fornax was seeing down from a 90 second stall to a 25 second stall. 25 seconds still isn't fantastic, but it's good enough to release, especially because Fornax has a pretty unique situation of 10,000 files totaling 10 TiB in a single directory, which prior to this MR was a pretty harsh worst-case for the repair loop.

Merge request reports