Expose report-type artifact data to frontend (internal API endpoint)
https://gitlab.com/gitlab-org/gitlab-ce/issues/46809 was merged which allows us to define a report and which files should go in that report.
We now need an internal API endpoint to grab the relevant JobArtifact.test_report
(JUnit, Selenium) files given a Build
and show it on the frontend (Merge Request, CI View).
This is different from https://gitlab.com/gitlab-org/gitlab-ce/issues/49265 as we need the JSON data available via some endpoint/API on the frontend instead of downloading the file.
JUnit
For JUnit, we need to do some parsing via Sidekiq and the format was already described here. Given the async parsing nature, this endpoint needs to be polled.
Internal API endpoint:
GET <namespace>/<project>/-/jobs/<job-iid>/reports/junit
Here describes how each attribute is converted between junit.xml and the payload of the internal API. The definition of junit.xml is described in http://llg.cubic.org/docs/junit/.
{ status_name: `started/parsing/finished/failed` status_reason: '...' // if status_name == 'failed', `status_reason` notifies why it failed count: { "total": "The total count of <testsuite tests="N">", "failed": "The total count of <testsuite failures="N">", "fixed": "The total count of fixed tests" }, results: [ { "name": "<testcase classname="X">", "description": "<testcase name="X">", "file": "<testcase file="X">", "duration": "<testcase time="X">", "status": "passed/failed/fixed", "status_message": "<failure message="xxx">" }, { "name": "<testcase classname="X">", "description": "<testcase name="X">", "file": "<testcase file="X">", "duration": "<testcase time="X">", "status": "passed/failed/fixed", "status_message": "<failure message="xxx">" }, ... ] }
https://gitlab.com/gitlab-org/gitlab-ce/issues/46809#note_87042469
Selenium
Thanks @dosuken123 for discussing!
# Returns the list of session ids
GET <namespace>/<project>/-/jobs/<job-id>/reports/selenium
# Raw selenium-log.json of the session id (e.g. `ba2773ceea765b6e40536a0e4381a31e`)
GET <namespace>/<project>/-/jobs/<job-id>/reports/selenium/:session_id
# Raw image of the screenshot id (e.g. `70dd4a0481-1537364414654`)
# Redirect to `<namespace>/<project>/builds/:id/artifacts/raw` endpoint (TODO: Expand for multiple reports)
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium_screenshots/:session_id/:screenshot_id
Previous iteration
JSON log
For Selenium, no parsing is needed. This report involves a wildcard, so we need a way to fetch each separate session log or combine them. We may want to combine to make the API format familiar across reports.
.gitlab-ci.yml
reports:
selenium: selenium/**/selenium-log.json
Internal API endpoint:
Combined session log example (option 1):
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium
{
status_name: `started/parsing/finished/failed`
status_reason: '...' // if status_name == 'failed', `status_reason` notifies why it failed
count: {}, // This was in the junit response but isn't necessary for Selenium. Perhaps should be a general `meta` object
results: {
31b6a0fb1403d76c6810811f12ce35a1: ...JSON log as-is...
<another-session-id>: ...JSON log as-is...
...
}
}
Separate session log example (option 2):
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium
[
"31b6a0fb1403d76c6810811f12ce35a1",
"<another-session-id>"
]
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/31b6a0fb1403d76c6810811f12ce35a1
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/<another-session-id>
...JSON log as-is...
Screenshots
In https://gitlab.com/gitlab-org/gitlab-ce/issues/46809#note_86604455, it was mentioned multiple times, that screenshots will be in their own zip file,
That internally (for runner) gets converted into the following artifacts definition:
selenium_logs: paths: [selenium/*/selenium-log.json], format: gzip, type: selenium_logs selenium_screenshots: paths: [selenium/*/*/screenshots/*.png], format: zip, type: selenium_screenshots
https://gitlab.com/gitlab-org/gitlab-ce/issues/46809#note_86811694
Internal API endpoint:
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/31b6a0fb1403d76c6810811f12ce35a1/screenshots/1612180645-1537364412515.png
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/<another-session-id>/screenshots/<other-screenshot-id>.png
As a note, we can already serve screenshots from the raw artifacts which is currently being used in the PoC with some hard-coded paths, https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6051
<namespace>/<project>/builds/3201/artifacts/raw/selenium/ba2773ceea765b6e40536a0e4381a31e/screenshots/1612180645-1537364412515.png
cc @ayufan @dosuken123 (separate thoughts welcome)