New RPCs: getblocktemplatelight and submitblocklight

Summary

GBT Light mining RPC added

Two new light-weight RPC calls were added: getblocktemplatelight and submitblocklight. These RPCs reduce the round-trip time for mining software when retrieving new block templates. Non-coinbase transaction data is never sent between mining software and bitcoind. Instead, job_id's are returned and bitcoind later reconstructs the full block based on this job_id and the solved header + coinbase tx submitted by the miner, leading to more efficient mining.

A full description and specification for this facility can be found here in this MR: !284 (merged) .

3 new CLI / config options were added to manage GBT Light:

• -gbtcachesize=<n> - Specify how many recent getblocktemplatelight jobs to keep cached in memory (default: 10). Jobs not in the memory cache will be loaded from disk.
• -gbtstoredir=<dir> - Specify a directory for storing getblocktemplatelight data (default: <datadir>/gbt/).
• -gbtstoretime=<secs> - Specify time in seconds to keep getblocktemplatelight data in the -gbtstoredir before it is automatically deleted (0 to disable autodeletion, default: 3600).

As usual, all of the above CLI arguments may also be specified in the .conf file for the node (but without the preceding - character).

Impact on existing subsystems

Minimal. The only thing to note is that a background task gets added to the global scheduler that bitcoind uses (iff started with nonzero -gbtstoretime which is the default). This scheduler is an app-wide facility for scheduling microtasks in a background thread. We added a task that runs once every 30 mins by default (and it completes in under 10ms under very heavy load in my testing -- under 0.1ms if there's little to do). This task is responsible for deleting old/stale data files from the gbtstoredir. Without this task the data directory could grow to fill the disk after a few weeks of mining.

Key change highlights

• Added gbtlight.cpp and gbtlight.h which initialize the subsystem. These files only get compiled into libserver.a, thus ensuring e.g. things like bitcoin-cli don't contain this code.

• Modified rpc/mining.cpp getblocktemplate and submitblock to support the two new RPCs.

• Wrote some unit tests which test the basic mechanism for the in-memory cache as well as the disk cache for managing job data. These live in test/gbtlight_tests.cpp

• Wrote two functional test cases for testing the end-to-end use of the new RPCs, testing error conditions, as well as testing the management of the gbtstoredir files (there is a background thread that cleans the files up after some time, which is tested by the functional tests).

• Added a new function in core_io: ParseHashStr for type uint160. This is the same as an existing overload that parses uint256. I also went ahead and wrote a unit test both for this and the existing uint256 version.

Motivation

Existing miners in China already have patches to ABC that implements this facility (not as efficiently as our implementation). The origins of this code were that patch, which has been massively reworked, optimized and redone to be faster and saner.

Test Plan

Basic:

1. ninja check
2. ninja check-functional
3. ./bitcoind -h

Medium:

Run bitcoind and hit it with bitcoin-cli help getblocktemplatelight and bitcoin-cli help submitblocklight

Advanced:

1. Run bitcoind and actually call getblocktemplatelight and see what it does. Perhaps feed it additional_txs, observe the appearance of files in the data directory.

2. Run bitcoind -gbtstoretime=20 (20 seconds) and hit the RPC with getblocktemplatelight and observe data files being expired first to gbt/trash/ in the data dir after 10 seconds of file age, and later being deleted altogether after 20 seconds of file age.

Merge Instructions

Kindly squash all commits. If you like, use this message as the commit message.