Skip to content

Tezt: change location of expected output files for regression tests

Romain requested to merge nomadic-labs/tezos:romain-tezt-regressions into master

Context

This MR is a prerequisite to be able to have multiple Tezt binaries spread accross the codebase (!5148 (merged)).

This MR changes the location of output files of regression tests. Before, files where in tezt/_regressions/<output_file>.out where tezt/_regressions could be controlled with --regression-dir and <output_file> was given when registering the test (with ~output_file). Now, files are in tezt/tests/expected/<file>/<title>.out where <file> is the name of the source file where the test is defined and <title> is the title of the test (with some sanitization). Parameter ~output_file and command-line parameter --regression-dir have been removed.

This MR changes the check controlled by --no-unknown-regression-files to not care about files in expected directories not related to registered tests. This allows multiple Tezt executables to coexist in the codebase.

This MR makes sure that tests can find files whether tests are run with dune runtest (in which case the current working directory is in _build) or with dune exec.

Manually testing the MR

The CI should pass, but you can run some quick regression tests with:

dune exec tezt/tests/main.exe -- -f config.ml

You can reset regressions with:

dune exec tezt/tests/main.exe -- -f config.ml --reset-regressions

You can also run the regression tests that are outside of tezt/tests:

cd src/proto_alpha/lib_protocol/test/regression
dune runtest
dune exec ./main.exe

(those tests are written in a way that assumes they are ran from src/proto_alpha/lib_protocol/test/regression, not because of regression files but because of contracts).

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Romain

Merge request reports