E2E: Show Job Stats in Slack and MR Report Summaries for Timed Out Jobs

Problem

Currently, when any E2E test job times out (such as e2e:saas, e2e:self-managed and e2e:saas-reliable), no test result file is generated in the artifacts, and the summary in Slack and merged MRs will not show any information about the job at all.

For example, we can see in this Slack summary for this pipeline that only e2e:saas-reliable and e2e:self-managed displays. e2e:saas had timed out:

Screenshot_2023-05-02_at_12.01.39_PM

This can be confusing and hide actual failures that need to be investigated.

Proposal

If a test result file is not generated, such as when a job times out, we should still show a summary for that job, ex:

JOB               | TOTAL | FAILURES | ERRORS | SKIPPED | RESULT
------------------|-------|----------|--------|---------|--------
e2e:saas          | N/A   | N/A      | N/A    | N/A     | No data
e2e:saas-reliable | 4     | 0        | 0      | 0       | Passed
e2e:self-managed  | 31    | 23       | 0      | 4       | Failed
Edited May 02, 2023 by Valerie Burton
Assignee Loading
Time tracking Loading