Skip to content

use TAP driver for tests

Currently we include several tests which are in fact a combination of many. For example:

  • tests/tls13-cert-key-exchange.c runs multiple TLS1.3 related tests using the gnutls test "framework"
  • tests/name-constraints-ip.c runs multiple name constraints related tests using the cmocka test framework

When running make check these tests show as one, and figuring out what failed requires running that test and examining its output.

However if we take advantage of the tap driver support of automake, we could display the "bundled" tests, and failures as part of the normal testing output. That would ease review of test suite output on failure, and would more accurately indicate the number of tests run. As such we should consider moving to TAP driver for output.

The most problematic tests for such a conversion are the internal framework tests; cmock tests can output TAP if CMOCKA_MESSAGE_OUTPUT=TAP is exported.

Example (before):

$ make TESTS=tls12-prf  check SUBDIRS=.
PASS: tls12-prf

with tap driver:

$ make TESTS=tls12-prf  check SUBDIRS=.
PASS: tls12-prf 1 - sha256_test1
PASS: tls12-prf 2 - sha256_test2
PASS: tls12-prf 3 - sha256_test3
PASS: tls12-prf 4 - sha256_test4
PASS: tls12-prf 5 - sha384_test1