Shell: adapt new mempool with proto add_operation
Context
The goal of this MR is to plug the protocol's Mempool.add_operation
, introduced in Lima, into the "new mempool" created in !6537 (merged).
To make this possible, we first do the following changes:
- in the shell:
- Extract the mempool operation representation and operation parsing from
prevalidation.ml
into a new fileshell_operation.ml
. Indeed, these are mostly independent from the purpose ofprevalidation.ml
, and they were the only reason thatprevalidator_classification.ml
depended onprevalidation.ml
. This allowsPrevalidation
to know aboutPrevalidator_classification.classification
instead. - Split the
Shell_plugin.FILTER
module type into a legacy version for protocols<= Kathmandu
and a new version for>= Lima
. This is similar to what !6537 (merged) did with the shell mempool. This allows us to make significant changes to the plugin interface to accommodate the new mempool. - Pass the plugin filter on as an argument to
Prevalidation.Make
(instead of only the protocol).
- Extract the mempool operation representation and operation parsing from
- in the plugin (for Lima and Alpha):
- Add a new function
conflict_handler
, that decides whether to replace an existing valid operation with a new conflicting one. It has the right type to be given as the eponymous argument to the protocol'sMempool.add_operation
. - Add a new function
add_operation_and_check_minimal_weight
to be called by the shell after validating each operation. This function is responsible for bounding the number of valid manager operations in the mempool. This function is intended to be remove in the future, when the logic for bounding the number (or maybe total size) of operations in the mempool gets moved from the plugin to the shell. But doing so now would significantly expand this already large MR.
- Add a new function
Then we can at last call Mempool.add_operation
from a new Prevalidation.add_operation
function, and call the later in the prevalidator_internal.ml
instead of the plugin's precheck
and post_filter
and Prevalation.apply_operation
. These obsolete functions are subsequently removed.
Many tests need to be updated as a result of these changes. Moreover, the old tests on Prevalidation.apply_operation
are heavily reworked into a test on Prevalidation.add_operation
, and new tests are added on the new plugin function Mempool.conflict_handler
.
This MR is mostly code engineering, and tries to minimize the impact on the mempool semantics. The only semantic changes are:
- Non-manager operations can now be replaced in the mempool if a new conflicting operation is higher according to the protocol's
Operation.compare
(non-manager operations used to never be replaced). Conflicts between two manager operations remain unchanged (i.e. the old operation is replaced iif the new one has both 5% better fees and a 5% betterfee/gas_limit
ratio). - New errors
Prevalidation.Operation_replacement
andPrevalidation.Operation_conflict
replace the plugin errors that could happen duringprecheck
. Note that despite these errors having different inherent categories (e.g.Temporary
) than the plugin ones, the operation will still receive the same classification as before (respectivelyOutdated
andBranch_delayed
). - Fix a bug that caused the mempool to accept a manager operation that conflicts with an already present
drain_delegate
operation. - In the specific case of the local injection of an operation batch whose total fees overflow, we now notice that the source cannot afford the fees before finding out about the overflow, resulting in a temporary error instead of a permanent one. (This is not mentioned in the changelog because it seems an unlikely corner case.)
Manually testing the MR
Run the following command to check updated prevalidation tests:
dune exec src/lib_shell/test/test_prevalidation_t.exe
Run the following command to check the newly added conflict handler tests:
dune exec src/proto_alpha/lib_plugin/test/test_conflict_handler.exe
CI green for other tests
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) - [N/A] Document any change to the user interface, including configuration parameters (see node configuration)
-
Provide automatic testing (see the testing guide). -
For new features and bug fixes, add an item in the appropriate changelog ( docs/protocols/alpha.rst
for the protocol and the environment,CHANGES.rst
at the root of the repository for everything else). -
Select suitable reviewers using the Reviewers
field below. -
Select as Assignee
the next person who should take action on that MR