Skip to content

Remove limit of 16 from -par

matricz requested to merge matricz/bitcoin-cash-node:unlimitpar into master

The current limit of 16 parallel script/sig verification threads is substantially increased.

Rationale

This limit was set 10 years ago when this functionality was first introduced. Since then both hardware and software have progressed vastly. Users should be able to set the parallelism higher if their hardware supports it.

In my own tests from a couple of years ago, and also reports from others, I tried to remove this limit and noticed that after ~16 threads there was indeed no improvement to verification time. Out of curiosity I re-ran these tests recently and found that, while there are diminishing returns, verification times do improve with more cores.

Changes

This changeset raises the maximum parallelism to a somewhat future-proof value of 256 threads, with the following rules:

  • For users that do not use the -par argument/config, current legacy functionality is preserved (clamped to 16)
  • For users that set -par=0, their number of cores will be used (clamped to 256)
  • For users that set -par=N, with N a positive integer, the parallelism will be set to N, clamped to 256; note that users can set this higher than their core count, which preserves current functionality
  • For users that set -par=N, with N a negative integer, the parallelism will be set to leave N threads free, clamped to 256
  • For users using the GUI, the maximum number of threads they can set is equal to the number of cores, and is no longer 15 (sic!)

Caveats

Current functionality is preserved only for users that don't use -par. Users that set -par=0 (and have more than 16 cores) will be affected, and we need to make that clear to them.

GUI users had the possibility of setting -par all the way up to 15 (instead of 16), which could be confusing. I made the decision to clamp the slider to GetNumCores() for GUI users. Headless users can still set -par above their core count.

An upper limit is still needed, otherwise users can create thousands of threads and crash the node.

Test plan

ninja all check-all
ninja bench_bitcoin
../contrib/bench/bench_par_validation.sh
Edited by matricz

Merge request reports