Skip to content

Draft: build: Generate junit output for boost unit tests

This diff makes the individual boost tests output a junit report XML, and merge them in order to produce a test suite junit report. The final report is located under a test/junit dir under the build directory.

This requires a few behavior changes and implies some limitations:

  • A new option ENABLE_JUNIT_REPORT (disabled by default) let the boost tests generate a Junit report, and proceed incremental merge of the results to a test suite report.
  • A test failure will stop the build process, and the result won't appear in the Junit report by default. One need to run ninja with the -k0 flag to change this behavior and allow the test suite to run to completion.
  • The test suite duration reported is not the actual duration, because it can't account for the tests running parallel. The reported duration is equivalent to running the tests serially.
  • The incremental nature of the report makes it impossible to clear the content by running the tests. Each test will override its previous run but not impact the other tests, independently of what is being run. To clear the report, ninja clean should be used.

Test Plan

cmake -GNinja ..
ninja check

Check no report is generated.

cmake -GNinja .. -DENABLE_JUNIT_REPORT=ON
ninja check

Check the report are generated.

ninja clean

Check the test/junit directory no longer exist.

Edit a test from the bitcoin test suite to make it fail:

ninja -k0 check

Ensure all the tests including the failure are correctly reported. Check the return code is non zero.

This MR also includes updates to .gitlab-ci.yml to make use of this feature. Review and observe CI jobs run correctly.

This is a backport of D7002

Edited by Dagur Valberg Johannsson

Merge request reports