Skip to content

Automated testing

Christoph Fischer requested to merge devel_automated_testing into master

This MR introduces a Python framework for automated testing of Met.3D configurations. Basically, it runs Met.3D in batch mode for each defined test instance (containing of pipeline and session files), generates a screenshot, and compares it to a "benchmark" screenshot. Therefore, we can check with every change in the code repository whether the "expected" visualization results are not altered by the code changes. This should make it easier to prevent bugs getting into a release version if we have an extensive testing set of data sets and sessions spanning over all supported level types, visualization techniques, property settings and so on.

The MR contains:

  • a main.py file which is the start point for the automated testing, this file calls the other .py files which contain helper methods,
  • the tests.json contains the meta description for each test (e.g., name, screenshot name, data set configuration, session),
  • a default frontend config used, but this one will be dynamically adapted to the test cases.

It can be run by editing the config.ini to point to the correct locations (e.g., the share drive and the Met3D executable), they can also be set via command line arguments, see python main.py --help.

More details are in a shared document on MS Teams.

Implements #367.

Edited by Christoph Fischer

Merge request reports