Alcotezt

Context

This is a proof of concept to ease the migration from Alcotest to Tezt. It provides a wrapper above Tezt with the same interface as Alcotest, so that our Alcotest tests can be run without Alcotest, but with Tezt instead.

With this wrapper, all you have to do to migrate a test to Tezt is to modify two letters in the manifest as such:

• replace test into tezt (well technically it can be tests into tezt so it can add one more character to the diff…);
• replace alcotest into alcotezt.

That's it. In the ideal case at least. For lib_base for instance, one more minor change was needed: replace filename with project_root // "src/lib_base" // filename. For lib_benchmark, it was also needed to declare the test module in the manifest.

It turns out that this even works for tests that use Qcheck_alcotest. We need to keep the qcheck_alcotest dependency for it to work but it shows that it should actually be straightforward to provide an equivalent module for Tezt.

This MR uses this approach for the tests of src/lib_error_monad. As you can see, none of the files in src/lib_error_monad (except automatically-generated ones) were modified. Same for src/lib_benchmark (after removing the unused file main.ml, which conflicted). Same for src/lib_base, but file paths needed to be adapted.

Because this is a proof of concept, it only contains a limited part of the Alcotest API. To migrate more tests, we'll need to provide all functions in the Alcotest module.

Logs

One issue is that some Octez tests print to stdout or stderr directly. We want Tezt to capture those so that it can put them in log files, and hide them by default.

Currently the Alcotezt library redirects Format.{std,err}_formatter to Tezt.Log.info. It works well, for instance it captures all that lib_benchmark outputs.

However it does not capture Stdlib.{std,err}out, which cannot be redirected (and which would create a recursive loop anyway, since the Log module logs on those channels). This means that Printf.{,e}printf and Stdlib.pr{int,err}_* are not captured.

In particular, when QCheck chooses a random seed it prints it using Printf.printf and this means that it is not captured. It's probably not verbose enough to care. I tried making a QCheck_alcotest module in the Alcotezt library but since some tests do not call QCheck_alcotest directly but call it through test helper libraries, it doesn't work unless we migrate all tests at once and thus can modify the test helper libraries.

Manually testing the MR

dune build @src/lib_error_monad/runtezt

Alternatively:

dune exec src/lib_error_monad/test/main.exe -- -v

or even:

# usual alias that you may already have
alias tezt='dune exec tezt/tests/main.exe --'
tezt -f error-registration -v

(error-registration is the string given to Alcotest.run, unfortunately this does not correspond to the filename, but we could also replace the strings in all calls to Alcotest.run into ~__FILE__).

Checklist

• Document the interface of any function added or modified (see the coding guidelines)
• Document any change to the user interface, including configuration parameters (see node configuration)
• Provide automatic testing (see the testing guide).
• For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
• Select suitable reviewers using the Reviewers field below.
• Select as Assignee the next person who should take action on that MR