Skip to content

WIP: Replace rtest ksh with pytest, addresses 1779

Bob Apthorpe requested to merge apthorpe/graphviz:rtest-1779 into master

test_rtest.py is a slightly cleaner modification of test_rtest2.py which runs the tests originally listed in rtest/tests.txt and compares the results. Unified diff listings are produced for text output, difference images are produced for image files (only tested with .png so far).

A few outstanding issues:

  • I've deferred adding text filtering until more tests pass and there's a better notion of what 'noise' should be filtered out
  • It would be nice to have standard default values (i.e. a naming scheme) of input and output directories plus a standard method of specifying the reference directory and the path to the dot executable.
  • rtest2/TESTS.json should be restructured to allow one script to handle both sets of tests; not sure it's wise to store literal graph input in JSON due to the amount of quote-escaping needed

This is marked as a WIP for two main reasons:

First, the format of tests.json needs review for general applicability. What else does it need? Ability to skip tests by platform? Fields for gitlab issues (references for regression tests)?

Second, it's still written as a standalone script that can be driven from pytest. I'm happy to more tightly integrate this with pytest but I need some guidance or sample code. As it, it works under pytest but the output is not nearly as terse or useful as the TAP output. A fair amount of hoop-jumping was needed to pass in the reference directory location and the path to the dot executable; there has to be a better way of driving this via pytest than what I'm doing.

Merge request reports