(2024Q1) - Layer 1 - RBT - DDB operations for blocks
DDB operations for blocks
This is a sub-milestone of %(2024Q1) - Layer 1 - Reduce Block time
The goal of this project is to improve the way block operations are fetched and avoid fetching from peers if the operations are already known by the mempool DDB.
People
Documents
Spec:
For each new block, the block producer advertised the block to its peers. Each of the peers needs to fetch the block operations from the block producer before validating it and advertise it to other peers.
These operations are stored in the block operations DDB. If the prevalidator of the node is running, most of these operations are already known in the prevalidator operations DDB.
Implementation
The new head message only contains the operation hash Merkel root, and not the operations hashes. The node needs to fetch all the block operations and reconstruct this Merkle root.
Rework
Instead of fetching all the operation, only fetch operations not already known in the prevalidator DDB.
First implementation:
- change the
new_head
message so that it contains the operations hashes. - nodes check what are the operations missing from its prevalidator.
- fetch only the missing operation with a new p2p message
get_block_operation_with_hash
- This solution adds more data in the
new_head
message since this message is advertised for each new head to each peer- benefits: no need for a new roundtrip message
- disadvantage: more data sent for each
new_head
messages
Second implementation:
- keep the
new_head
message as it is - introduce a
get_block_operation_hashes
message that a node send to only one peer (instead of fetching block operations) - Once the answer
block_operation_hashes
is received, the node check what are the hashes missing from its prevalidator DDB. - fetch from the peer with
get_block_operation_with_hash
only the operations missing. - This solution adds new messages before fetching
- benefits: if no operations need to be fetched, only one
get_block_operation_hashes
andblock_operation_hashes
is sent after anew_head
message - disadvantage: a new roundtrip message is introduced if every operation needs to be fetched
- benefits: if no operations need to be fetched, only one
Overall Benefits:
- Reduce the bandwidth usage
- Not yet measured
- Reduce the latency
- Not yet measured
Work breakdown
-
implement first solution -
validate with experiment the gains -
improve overall latency -
reduce bandwidth consumption
-
-
-
implement second solution (!12423) -
validate with experiment the gains (experiments results) -
improve overall latency -
reduce bandwidth consumption
-
-
-
Introduce a p2p versioning that allow legacy and new solution (the best one from validation results) -
For peers using the legacy version, continue requesting / sending all block operations -
For peers using the new version, only request unknown operations
-
-
Add tests -
ensure that a node can connect to 2 nodes, one using the legacy version and one using the new version
-