Better testability, more tests
Mission
Make certain parts of the codebase more easily testable and add tests that enhance safety. Make small iterative changes, perform refactorings, decouple implementations from their backends, and split implementations to make code easier to manage. This will allow to write unit tests, property tests, and integration tests; hereby enhancing safety. New tests must be automatically run on the CI.
- There are things that cannot be linked on this milestone, because it lives on
tezos/tezos
. There's another milestone onnomadic-labs/tezos
here to circumvent this issue.
Current state
At the moment, some libraries are hard to test, for a range of reasons:
- Files and functions are too big.
- Implementation is coupled to a concrete backend, making mocking parts of the codebase difficult or impossible.
As a consequence:
- It makes it difficult to write tests when fixing the concerned libraries. So some merge requests are merged without tests which is sad
😿 and dangerous - When a security issue is found, it is sometimes harder to reproduce than it should be.
Objectives
- Continue this refactoring/abstraction/decoupling/testing process on a few core libraries:
lib_shell
andlib_p2p
. So far we have been guided by the bugs found by Protest2k and the bug found by handy octopus; and then by bugs reported/found by @Saroupille. We have still one bug in https://gitlab.com/nomadic-labs/tezos/-/issues/476 to reproduce, and the private bootstrap pipeline one. - Fuzz public APIs, such as the node's RPC server and the distributed db's interface. We envision no technical difficulty for qchecking (i.e. fuzzing) with tezt the node's RPC server. We will have results in a few weeks. Then two roads will be possible: either 1/ ensure that this coverage is always complete, by fuzzing the entire RPC directory, using its programmatic description (
/describe
); instead of fuzzing a hand-picked subset of the RPCs. 2/ instead of testing externally, test internally by fuzzing the mockup's RPC server. This internal fuzzing would be much faster because there would be no network involved.
Ongoing work
Our efforts are currently focusing on making the mempool more tested (src/lib_shell/prevalidator.ml
) (@sir4ur0n and @smelc), lib_p2p
(@sir4ur0n and @smelc), and to fuzz the RPC server (@Divesh-Otwani).
We refer to the issues and merge requests linked to this milestone for detailed progress.
Acceptance Criteria
- Low-hanging-fruit changes are exhausted.
- Fuzzing described above is complete.