Manifest + Tezt = easy to define local Tezt tests
Context
The idea is to be able to define Tezt tests in libraries, in particular unit tests.
Devs would just:
- create test files like
src/lib_stdlib/tezt/test_stdlib.ml
; - use
Test.register
in those files; - declare those tests in the manifest with the
tezt
function of moduleManifest
.
Then one could use dune build @src/lib_stdlib/runtezt
to run those tests and get something like:
tezos_stdlib_test_something_tezt_exe alias src/lib_stdlib/tezt/runtezt
[13:44:53.708] [SUCCESS] (1/1) dummy test for tezos-stdlib (something)
tezos_stdlib_test_stdlib_tezt_exe alias src/lib_stdlib/tezt/runtezt
[13:44:53.719] [SUCCESS] (1/1) dummy test for tezos-stdlib
Or one can also run the test from the main Tezt executable which gathers all tests.
Each test only depends on the dependencies that are declared in the manifest + the tezt
library, i.e. they can be quick to compile if they don't depend on, say, tezos-base
. For unit tests that do not depend on tezt-tezos
for instance, it should result in faster compilation times.
runtezt
Alias
The In order not to run Tezt tests in the CI twice (once with dune runtest src/lib_X
and once with dune exec tezt/tests/main.exe
), Tezt tests are not added to the runtest
alias. They are added to the runtezt
alias instead. Which means that one cannot run dune runtest src/lib_X
to run Tezt tests: one has to run dune build @src/lib_X/runtezt
.
Additionally, for packages which have Tezt tests, we add dune exec runtezt {with-test}
to the .opam
files.
Having a different command (runtest
vs build @runtezt
) is not ideal, but it was the easiest way I could find to exclude Tezt tests from the CI unit tests.
Caveat
Tests in src/proto_alpha
will eventually be duplicated during the snapshot, and there is some code to make sure their ~title
will change when this happens. This piece of code should probably, eventually, be put somewhere where it can be shared among other protocol tests, maybe even enforced somehow.
Manually testing the MR
This MR ported src/proto_alpha/lib_protocol/test/regression
. To test it:
make -C manifest
# There is a runtezt alias.
dune build @src/proto_alpha/lib_protocol/test/regression/runtezt
# You can also just run the executable (this allows to pass command-line arguments).
dune exec src/proto_alpha/lib_protocol/test/regression/main.exe
# Tests are also available from "the big Tezt executable".
# This is how the CI runs tests, to benefit from auto-balancing.
dune exec tezt/tests/main.exe -- -f test_logging.ml
# Also works from other directories.
cd src/proto_alpha/lib_protocol/test/regression
dune build @runtezt
dune exec ./main.exe
You should also check that the CI actually runs the test_logging.ml
tests.
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) -
Document any change to the user interface, including configuration parameters (see node configuration) -
Provide automatic testing (see the testing guide). -
For new features and bug fixes, add an item in the appropriate changelog ( docs/protocols/alpha.rst
for the protocol and the environment,CHANGES.rst
at the root of the repository for everything else). -
Select suitable reviewers using the Reviewers
field below. -
Select as Assignee
the next person who should take action on that MR