Skip to content

Shell: adapt new mempool with proto add_operation

Context

The goal of this MR is to plug the protocol's Mempool.add_operation, introduced in Lima, into the "new mempool" created in !6537 (merged).

To make this possible, we first do the following changes:

  • in the shell:
    • Extract the mempool operation representation and operation parsing from prevalidation.ml into a new file shell_operation.ml. Indeed, these are mostly independent from the purpose of prevalidation.ml, and they were the only reason that prevalidator_classification.ml depended on prevalidation.ml. This allows Prevalidation to know about Prevalidator_classification.classification instead.
    • Split the Shell_plugin.FILTER module type into a legacy version for protocols <= Kathmandu and a new version for >= Lima. This is similar to what !6537 (merged) did with the shell mempool. This allows us to make significant changes to the plugin interface to accommodate the new mempool.
    • Pass the plugin filter on as an argument to Prevalidation.Make (instead of only the protocol).
  • in the plugin (for Lima and Alpha):
    • Add a new function conflict_handler, that decides whether to replace an existing valid operation with a new conflicting one. It has the right type to be given as the eponymous argument to the protocol's Mempool.add_operation.
    • Add a new function add_operation_and_check_minimal_weight to be called by the shell after validating each operation. This function is responsible for bounding the number of valid manager operations in the mempool. This function is intended to be remove in the future, when the logic for bounding the number (or maybe total size) of operations in the mempool gets moved from the plugin to the shell. But doing so now would significantly expand this already large MR.

Then we can at last call Mempool.add_operation from a new Prevalidation.add_operation function, and call the later in the prevalidator_internal.ml instead of the plugin's precheck and post_filter and Prevalation.apply_operation. These obsolete functions are subsequently removed.

Many tests need to be updated as a result of these changes. Moreover, the old tests on Prevalidation.apply_operation are heavily reworked into a test on Prevalidation.add_operation, and new tests are added on the new plugin function Mempool.conflict_handler.

This MR is mostly code engineering, and tries to minimize the impact on the mempool semantics. The only semantic changes are:

  • Non-manager operations can now be replaced in the mempool if a new conflicting operation is higher according to the protocol's Operation.compare (non-manager operations used to never be replaced). Conflicts between two manager operations remain unchanged (i.e. the old operation is replaced iif the new one has both 5% better fees and a 5% better fee/gas_limit ratio).
  • New errors Prevalidation.Operation_replacement and Prevalidation.Operation_conflict replace the plugin errors that could happen during precheck. Note that despite these errors having different inherent categories (e.g. Temporary) than the plugin ones, the operation will still receive the same classification as before (respectively Outdated and Branch_delayed).
  • Fix a bug that caused the mempool to accept a manager operation that conflicts with an already present drain_delegate operation.
  • In the specific case of the local injection of an operation batch whose total fees overflow, we now notice that the source cannot afford the fees before finding out about the overflow, resulting in a temporary error instead of a permanent one. (This is not mentioned in the changelog because it seems an unlikely corner case.)

Manually testing the MR

Run the following command to check updated prevalidation tests:

dune exec src/lib_shell/test/test_prevalidation_t.exe

Run the following command to check the newly added conflict handler tests:

dune exec src/proto_alpha/lib_plugin/test/test_conflict_handler.exe

CI green for other tests

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • [N/A] Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Albin Coquereau

Merge request reports