Skip to content
Tags give the ability to mark specific points in history as being important
  • 3.9 protected
    246683e0 · Prepare release 3.9 ·
    Release: Release 3.9
    Release 3.9
    - Option `--format` creates a temporary copy of the input program and formats it with `clang format`.
      The internal program transformations of TestCov work best with the formatting style of `clang format`.
    - TestCov now generates a temporary file ensuring the presence of EOF.
      This is necessary to abort test-execution.
  • 3.8 protected
    5629ac0c · Prepare release 3.8 ·
    Release: Release 3.8
    Release 3.8
    - Maintenance release (bug fixes, improved tests)
  • 3.7 protected
    f71d5de1 · Prepare release 3.7 ·
    Release: Release 3.7
    Release 3.7
    - Verbose logging only verbosely logs messages of TestCov, not of other subcomponents.
      Especially matplotlib debug messages are not displayed anymore when command-line flag
      '--verbose' is used. This reduces clutter in the verbose output.
    - The log prefix 'DEBUG:root' changed to 'DEBUG:suite_validation'.
    - Nicer-looking progress display: Log messages and the progress display
      now work nicely together and do not intermingle, and the progress display shows
      the currently executed and total number of tests.
      Example: "⏳ Executing tests  20/227"
  • v3.6 protected
    3fd90817 · Prepare version 3.6 ·
    Version 3.6
    - Report total number of tests found in a given test suite.
      We previously only reported the number of executed tests, but it may be interesting
      to see whether all identified tests were executed.
    - A keyboard interrupt (SIGINT, usually Ctrl+C) does not display an exception anymore,
      but gracefully terminates.
      On an interrupt during test-suite execution, execution will stop
      and result files will be produced.
      It is then possible to abort the generation of result files by interrupting again.
    - Get correct coverage measurement for single-line functions that call __assert_fail.
      A gcov_dump to dump recorded coverage was missing for these cases.
  • v3.5 protected
    aa1eb0a7 · Prepare version 3.5 ·
    Release: Version 3.5
    Version 3.5
    - Improved speed and supported language features for test-goal instrumentation in TestCov.
      We've replaced the Python-based implementation of adding test goals to the program
      with an implementation based on clang libtooling.
      This allows us to support the full range of C language features and GCC extensions.
      Before, we had to abort if unsupported features were used; most notably variadic arguments,
      but also GCC extensions that we did not remove explicitly.
    - Removed a bug that made TestCov fail if preprocessed programs
      contained method declarations of builtin methods with the
      Star operator '*' right before the method name (e.g., '*malloc').
      Such preprocessed programs are now fully supported.
  • v3.4 protected Release: Version 3.4
    Version 3.4
    - Always add calls to __gcov_flush before abort and __assert_fail calls.
      Before, this only happened for coverage goals "branch coverage" and "function-call coverage".
      This means that coverage measurements were not stored
      for condition and line coverage if an assert or abort was hit.
    - TestCov and its dependencies can now be installed with `pip install .`
    - Changed __gcov_dump to __gcov_flush (__gcov_dump does not exist anymore in GCC 11).
    - Support new naming scheme and directories for GCDA and GCOV files starting with GCC 11.
    - During code transformation, TestCov now always removes preprocessor comments so that
      the original file name is not used in GCOV output (starting with GCC 11), but the name of the transformed file.
      This avoids user confusion in the output.
    - Removed unnecessary semicolons after compiler directives in test harness.
    - Improved error messages for missing files and on gcov and lcov errors.
    - Cosmetic change: Call gcc with `--coverage` instead of `-fprofile-arcs -ftest-coverage`.
    - For testing, switched from nosetest to pytest
    - Store all metadata in setup.cfg instead of
      (easier to read and can contain more information about other tools)
    - Minor code improvements
  • v3.3 protected Release: Version 3.3
    Version 3.3: Bug squashing in program instrumentation
  • v3.2 protected
    Version 3.2: Reduce number of labels injected into program code for measuring branch coverage
  • v3.1 protected
    Version 3.1: Vastly improved program instrumentation for coverage and speed improvements in coverage computation
  • testcomp20 protected Release: Test-Comp 2020 release
    Version used in Test-Comp'20
  • v3.0 protected
    5c65f239 · Add numpy as dependency ·
    Version 3.0: Rename and new features
    * Rename test-suite validator to TestCov
    (`bin/testsuite-validator` is now `bin/testcov`)
    * Add additional test-suite reduction strategy based on count distance
    between coverages
    * Merge the two plots created for accumulated and individual coverage
    into single plot
  • v2.0 protected
    be7448e7 · Fix name ·
    Version 2 of testsuite-validator
    * Runs tests in a container through runexec.
      You can use `--memlimit MEM` and `--cpu-cores #CORES`
      to specify these limits (e.g.: `--memlimit 2GB --cpu-cores 2`)
      This requires access to cgroups (cf.
      If it is not possible for you to provide this access,
      use argument `--no-runexec`. With this parameter,
      testsuite-validator will only use containerexec,
      but no cgroups.
    * Creates not only coverage information for line and branch coverage,
      but also for condition coverage.
      Previously, the number of evaluated conditions
      was shown ('branches executed'), but not the conditions taken.
      'Branches executed' is not displayed anymore because we don't see
      any usage for it.
    * Coverage goals are provided through property files in Test-Comp
      format (
      If the goal is 'coverage-error', test-suite execution stops after
      a covering test is found.
      Argument `--stop-after-success` is replaced by this.
    * The coverage to display in coverage statistics is chosen
      according to the coverage goal provided.
    * Creates additional coverage statistics for individual tests
      and plots for this data. To turn any of these off,
      use argument `--no-sequence-file`, `--no-individual-test-coverage`,
      and `--no-plots`.
    * Creates a reduced test suite not only for coverage goal coverage-error,
      but also for line, branch and condition coverage.
      To disable this, use argument `--no-create-reduced-suite`
  • v1.3 protected
    First standalone version of tbf-testsuite-validator.
    Identical in features to v1.2, but with small adjustments
    to be independent of other test-format tools.
    Previous project tbf-testsuite-validator belonged to: