WIP: Run unit tests, regression tests, and CI tests via CMake/CTest, addresses 1772
This MR has three main goals:
- configure CMake to run unit tests, regression tests, and CI tests via CTest as mentioned in #1772 (closed)
- build unit tests against the Criterion test framework (see https://github.com/Snaipe/Criterion), either by using a a pre-existing installation or downloading, building, and locally installing a fresh copy, and
- replace the deprecated
find_package(PythonInterp)
withfind_package(Python)
The focus is on driving the existing tests from CMake so no effort has been put into fixing or upgrading tests. Regression tests are the most successful, completely passing during local testing on Windows and Linux with one failure (test_long_chain
) on OSX. The unit test executable command_line
built on OSX and Windows but failed several subtests; the executable did not build on Linux (link error for unresolved symbol agerr
which I believe is from cgraph
). Regardless, building against Criterion seemed to work on all platforms as did building an external copy of Criterion itself.
Note that the functionality to download, build, and install Criterion is limited to cmake/BuildCriterion.cmake
; this can be disabled in the main CMakeLists.txt
file by commenting out include(BuildCriterion)
Several tests required the use of python
or python3
but did not specify the executable to be used for testing. find_package(PythonInterp)
was replaced with find_package(PythonInterp COMPONENTS Interpreter)
and the variable Python_EXECUTABLE
was substituted for both python
and python3
for consistency in running tests.
This addresses many of the infrastructure issues which hampered comprehensive testing under CMake and hopefully lets people focus on revising the tests and application code instead of the build and test infrastructure.