Skip to content

Generate a machine-readable test status summary report

Current situation

Workflows are often series of tests. We produce an Allure report, but this is for human's consumption. There is a need for a machine-readable report, representing the overall tests execution, so that we can integrate better with CI tools such as the test report pane in GitLab MRs (and the corresponding feature in other CI toolchains).

Desired outcome

There should be a global execution report produced after (at least) successful workflow completion that summarize the test execution status.

(The executionlog service could be used as an inspiration.)

This report should be a Surefire-like report, in XML. It should include the test names and their execution status as well as their execution time.

Initially, we are targeting Robot Framework and Cucumber JVM tests.

The produced report should be understandable by GitLab test view.