Release 3.8

- Maintenance release (bug fixes, improved tests)

Release 3.7

- Verbose logging only verbosely logs messages of TestCov, not of other subcomponents.
  Especially matplotlib debug messages are not displayed anymore when command-line flag
  '--verbose' is used. This reduces clutter in the verbose output.

- The log prefix 'DEBUG:root' changed to 'DEBUG:suite_validation'.

- Nicer-looking progress display: Log messages and the progress display
  now work nicely together and do not intermingle, and the progress display shows
  the currently executed and total number of tests.
  Example: "⏳ Executing tests  20/227"

Version 3.6

Added

- Report total number of tests found in a given test suite.
  We previously only reported the number of executed tests, but it may be interesting
  to see whether all identified tests were executed.

Fixed

- A keyboard interrupt (SIGINT, usually Ctrl+C) does not display an exception anymore,
  but gracefully terminates.
  On an interrupt during test-suite execution, execution will stop
  and result files will be produced.
  It is then possible to abort the generation of result files by interrupting again.
- Get correct coverage measurement for single-line functions that call __assert_fail.
  A gcov_dump to dump recorded coverage was missing for these cases.

Version 3.5

Changed

- Improved speed and supported language features for test-goal instrumentation in TestCov.
  We've replaced the Python-based implementation of adding test goals to the program
  with an implementation based on clang libtooling.
  This allows us to support the full range of C language features and GCC extensions.
  Before, we had to abort if unsupported features were used; most notably variadic arguments,
  but also GCC extensions that we did not remove explicitly.

Fixed

- Removed a bug that made TestCov fail if preprocessed programs
  contained method declarations of builtin methods with the
  Star operator '*' right before the method name (e.g., '*malloc').
  Such preprocessed programs are now fully supported.

Version 3.4

Added

- Always add calls to __gcov_flush before abort and __assert_fail calls.
  Before, this only happened for coverage goals "branch coverage" and "function-call coverage".
  This means that coverage measurements were not stored
  for condition and line coverage if an assert or abort was hit.
- TestCov and its dependencies can now be installed with `pip install .`

Changed

- Changed __gcov_dump to __gcov_flush (__gcov_dump does not exist anymore in GCC 11).
- Support new naming scheme and directories for GCDA and GCOV files starting with GCC 11.
- During code transformation, TestCov now always removes preprocessor comments so that
  the original file name is not used in GCOV output (starting with GCC 11), but the name of the transformed file.
  This avoids user confusion in the output.
- Removed unnecessary semicolons after compiler directives in test harness.
- Improved error messages for missing files and on gcov and lcov errors.
- Cosmetic change: Call gcc with `--coverage` instead of `-fprofile-arcs -ftest-coverage`.
- For testing, switched from nosetest to pytest
- Store all metadata in setup.cfg instead of setup.py.
  (easier to read and can contain more information about other tools)
- Minor code improvements

Version 3.3: Bug squashing in program instrumentation

Version 3.2: Reduce number of labels injected into program code for measuring branch coverage

Version 3.1: Vastly improved program instrumentation for coverage and speed improvements in coverage computation

Version used in Test-Comp'20

Version 3.0: Rename and new features

* Rename test-suite validator to TestCov
(`bin/testsuite-validator` is now `bin/testcov`)
* Add additional test-suite reduction strategy based on count distance
between coverages
* Merge the two plots created for accumulated and individual coverage
into single plot

Version 2 of testsuite-validator

* Runs tests in a container through runexec.
  You can use `--memlimit MEM` and `--cpu-cores #CORES`
  to specify these limits (e.g.: `--memlimit 2GB --cpu-cores 2`)
  This requires access to cgroups (cf.
https://github.com/sosy-lab/benchexec/blob/master/doc/INSTALL.md#setting-up-cgroups)
  If it is not possible for you to provide this access,
  use argument `--no-runexec`. With this parameter,
  testsuite-validator will only use containerexec,
  but no cgroups.

* Creates not only coverage information for line and branch coverage,
  but also for condition coverage.
  Previously, the number of evaluated conditions
  was shown ('branches executed'), but not the conditions taken.
  'Branches executed' is not displayed anymore because we don't see
  any usage for it.

* Coverage goals are provided through property files in Test-Comp
  format (https://test-comp.sosy-lab.org/2019/rules.php).
  If the goal is 'coverage-error', test-suite execution stops after
  a covering test is found.
  Argument `--stop-after-success` is replaced by this.

* The coverage to display in coverage statistics is chosen
  according to the coverage goal provided.

* Creates additional coverage statistics for individual tests
  and plots for this data. To turn any of these off,
  use argument `--no-sequence-file`, `--no-individual-test-coverage`,
  and `--no-plots`.

* Creates a reduced test suite not only for coverage goal coverage-error,
  but also for line, branch and condition coverage.
  To disable this, use argument `--no-create-reduced-suite`

First standalone version of tbf-testsuite-validator.

Identical in features to v1.2, but with small adjustments
to be independent of other test-format tools.

Previous project tbf-testsuite-validator belonged to:

https://gitlab.com/sosy-lab/software/test-format/