Skip to content

CMake: upgrade test suite to Catch2 v3

Matthew Fernandez requested to merge smattr/graphviz:smattr/b9335b08-90b2-41e1-b57b-3cfa2cf10acf into main

Catch2 contains some breaking changes in the transition from v2 → v3. These are now packaged in Ubuntu 24.04, so upgrading the CI ctest job (which this commit does) required forward porting the test suite. Compatibility with Catch2 v2 is not preserved. Thus this change also removes building of the ctest test suite from prior versions of Ubuntu in CI.

Some more information:

A number of other changes were necessary within this commit:

  1. Generated files, in addition to being excluded with --exclude=… as they were already, are now also excluded from gcovr analysis with --gcov-exclude=…. It is unclear to me why both are needed on Ubuntu 24.04 but without this coverage analysis fails with many repeated messages:
          In directory /:
    /builds/smattr/graphviz/cmd/tools/gmlparse.y:source file is newer than notes file 'builds/smattr/graphviz/build/cmd/tools/CMakeFiles/gml2gv.dir/gmlparse.c.gcno'
    (the message is displayed only once per source file)
    /builds/smattr/graphviz/cmd/tools/../../lib/cgraph/stack.h:source file is newer than notes file 'builds/smattr/graphviz/build/cmd/tools/CMakeFiles/gml2gv.dir/gmlparse.c.gcno'
    /builds/smattr/graphviz/cmd/tools/../../lib/cgraph/agxbuf.h:source file is newer than notes file 'builds/smattr/graphviz/build/cmd/tools/CMakeFiles/gml2gv.dir/gmlparse.c.gcno'
    /builds/smattr/graphviz/cmd/tools/../../lib/cgraph/alloc.h:source file is newer than notes file 'builds/smattr/graphviz/build/cmd/tools/CMakeFiles/gml2gv.dir/gmlparse.c.gcno'
    /builds/smattr/graphviz/cmd/tools/../../lib/cgraph/exit.h:source file is newer than notes file 'builds/smattr/graphviz/build/cmd/tools/CMakeFiles/gml2gv.dir/gmlparse.c.gcno'
    Cannot open source file /builds/smattr/graphviz/build/cmd/tools/gmlparse.c

            (gcovr could not infer a working directory that resolved it.)
    To ignore this error use option --gcov-ignore-errors=no_working_dir_found.

  1. The --gcov-ignore-errors=no_working_dir_found option has been added to the gcovr invocation. It is unclear to me why but on Ubuntu 24.04 even with both --exclude=… and --gcov-exclude=… gcovr tries to process the Qt-generated file qrc_mdi.cpp and errors out on it.

  2. libtest_common is no longer linked against Catch2. On Ubuntu 24.04, Catch2 only ships as a static library (.a). I do not know why this is, but I suspect this is deliberate to avoid the main function that the library now provides being multiply loaded. A side effect of this is that libtest_common (a shared library) cannot link against it. Any attempt to make libtest_common an object or static library exceeds disk space and/or artifact limits in Gitlab’s CI environment. So we compromise and link each eventual test executable against Catch2 instead. This leaves the minor foot gun that libtest_common is #including Catch2 headers without Catch2’s include path being taken into account, but this seems to be the best we can do as the FindCatch2.cmake shipped with Catch2 does not expose a variable for its headers’ location.

Gitlab: closes #2465 (closed), closes #2499 (closed)

Merge request reports