Workflows are often series of tests. We produce an Allure report, but this has not changed from the early days and is poorly accessible, viewable, and extensible.
Desired outcome
There should be a global execution report produced after (at least) successful workflow completion that summarize the test execution status.
(The executionlog service could be used as an inspiration.)
There should at least be a summary (doughnut or some other overview) of the test statuses, a count of passed tests, and, for each test, its status, duration, and, if appropriate, its execution details.
A summary of the environment variables would be a plus. Ditto for execution environment's tags and technologies.
It should also contains the start and end date, as well as the total execution duration.
The Insight Collector service now generates, upon a workflow completion, an execution report in HTML format.
By default (if no SummaryReport insight is specified in the configuration file or if the Insight Collector service is not configured), this HTML report has four blocks:
1) Summary block:
a subtitle containing the workflow start, end and execution time;
a test execution summary doughnut with legend;
if the workflow contains multiple job, a test execution summary doughnut for each job;
a testcases by technology doughnut with legend.
2) Jobs cloud:
a subtitle with executed jobs total count;
a jobs cloud displaying job titles and legend explaining job titles color codes. Mouse over a job name shows a tooltip window listing job tags, OK and KO test cases summary for this job and environment variables related to this job (if any).
3) Tags table:
A table with Tag, OK, KO, Other and Total columns, sortable and filterable, showing test execution results by execution environment tag.
4) Testcases table:
a subtitle with test execution global statistics,
a table with Test suite, Testcase, Dataset, Status, Technology and Job columns, sortable and filterable. Testcase rows are clickable, the popup window that shows up contains a sort of workflow step restitution with name, uses, runs-on, managed, reference and data sections.
If test is not managed by SquashTM (managed: false), reference and data sections are not displayed. Otherwise, reference section contains test reference and data section the CUFs. Dataset column displays, for managed tests, a dataset name (if any). It is a clickable field, the popup window that shows up contains the dataset variables.
In the sortable and filterable tables, the table can be sorted on each column in ascending and descending order, and the filter is applied to all table fields.