Unit Test report documentation gaps
Problem to solve
As a developer who wants Test Report data in the GitLab UI, I want clear documentation about what parts of test reports are parsed, important, etc, so that I can implement an exporter / put data into my test cases that will be relevant and actionable to users debugging.
Some of the problems from the original write-up (see below).
- Update link on Unit Test Report docs for IBM to be https://www.ibm.com/support/knowledgecenter/SSQ2R2_14.2.0/com.ibm.rsar.analysis.codereview.cobol.doc/topics/cac_useresults_junit.html
- I don't know what parts of the JUnit file will be parsed. For example
<failure>
is not parsed currently. Being explicit about what is parsed and where it appears in the UI would be very helpful. - The "Use Case" and "Overview" sections of the documentation are redundant. The Use Case could be extended to how users can make use of the test summary from MR to MR as well.
The Wikipedia link in the "How to set it up" section is no longer relevant/helpful.
Further details
Having an example junit.xml file mapped to the GUI may be a very helpful way to explain this feature, especially if a user is trying to write their own exporter.
Proposal
Make changes as noted in the problems.
Who can address the issue
Pretty much anyone.
Other links/references
I realize somebody implementing export functionality in a new test framework is not a typical user story, so I understand if the documentation will not go in depth. An example of a 'GitLab-optimized' export file would be very nice however. Finally, please note: this feature is great! https://gitlab.com/gitlab-org/gitlab-ce/issues/17081 @dosuken123 You worked on https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/21003 which introduced this, could you have a look? Otherwise @fabiopitino, you added https://gitlab.com/gitlab-org/gitlab-ce/commit/24a81b3dccef87cf4c6837a26fadd702b0418d72?
Original Proposal
<failure>
are key-value pairs separated by a colon, while in fact GitLab allows you to put anything there (and does not parse such key-value pairs). (So I spent some time getting my system to spit out filename and line number, which got quite hairy, and turned out to be pointless classname
attribute which will somehow be shown (https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/28376), but it is not clear to me what it is meant for (I cannot find it in the docs). There are some files like test_case_entity.rb
, test_case.json
, and state.js
, which seem to contain lists of different keys, but how they are handled does not become clear. The Q&A on https://stackoverflow.com/q/442556 contains some links which are more complete than the IBM spec, perhaps they are helpful.
Related
Who can address the issue