Expose report-type artifact data to frontend (internal API endpoint)

https://gitlab.com/gitlab-org/gitlab-ce/issues/46809 was merged which allows us to define a report and which files should go in that report.

We now need an internal API endpoint to grab the relevant JobArtifact.test_report(JUnit, Selenium) files given a Build and show it on the frontend (Merge Request, CI View).

This is different from https://gitlab.com/gitlab-org/gitlab-ce/issues/49265 as we need the JSON data available via some endpoint/API on the frontend instead of downloading the file.

JUnit

For JUnit, we need to do some parsing via Sidekiq and the format was already described here. Given the async parsing nature, this endpoint needs to be polled.

Internal API endpoint:

GET <namespace>/<project>/-/jobs/<job-iid>/reports/junit

Here describes how each attribute is converted between junit.xml and the payload of the internal API. The definition of junit.xml is described in http://llg.cubic.org/docs/junit/.

{
  status_name: `started/parsing/finished/failed`
  status_reason: '...' // if status_name == 'failed', `status_reason` notifies why it failed 
  count: { "total": "The total count of <testsuite tests="N">", "failed": "The total count of <testsuite failures="N">", "fixed": "The total count of fixed tests" },
  results: [
      {
        "name": "<testcase classname="X">",
        "description": "<testcase name="X">",
        "file": "<testcase file="X">",
        "duration": "<testcase time="X">",
        "status": "passed/failed/fixed",
        "status_message": "<failure message="xxx">"
      },
      {
        "name": "<testcase classname="X">",
        "description": "<testcase name="X">",
        "file": "<testcase file="X">",
        "duration": "<testcase time="X">",
        "status": "passed/failed/fixed",
        "status_message": "<failure message="xxx">"
      },
      ...
  ]
}

https://gitlab.com/gitlab-org/gitlab-ce/issues/46809#note_87042469

Selenium

Thanks @dosuken123 for discussing!

# Returns the list of session ids
GET <namespace>/<project>/-/jobs/<job-id>/reports/selenium

# Raw selenium-log.json of the session id (e.g. `ba2773ceea765b6e40536a0e4381a31e`)
GET <namespace>/<project>/-/jobs/<job-id>/reports/selenium/:session_id 

# Raw image of the screenshot id (e.g. `70dd4a0481-1537364414654`)
# Redirect to `<namespace>/<project>/builds/:id/artifacts/raw` endpoint (TODO: Expand for multiple reports)
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium_screenshots/:session_id/:screenshot_id
Previous iteration

JSON log

For Selenium, no parsing is needed. This report involves a wildcard, so we need a way to fetch each separate session log or combine them. We may want to combine to make the API format familiar across reports.

.gitlab-ci.yml

reports:
  selenium: selenium/**/selenium-log.json

Internal API endpoint:

Combined session log example (option 1):

GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium

{
  status_name: `started/parsing/finished/failed`
  status_reason: '...' // if status_name == 'failed', `status_reason` notifies why it failed 
  count: {}, // This was in the junit response but isn't necessary for Selenium. Perhaps should be a general `meta` object
  results: {
      31b6a0fb1403d76c6810811f12ce35a1: ...JSON log as-is...
      <another-session-id>: ...JSON log as-is...
      ...
  }
}

Separate session log example (option 2):

GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium
[
  "31b6a0fb1403d76c6810811f12ce35a1",
  "<another-session-id>"
]

GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/31b6a0fb1403d76c6810811f12ce35a1
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/<another-session-id>
...JSON log as-is...

Screenshots

In https://gitlab.com/gitlab-org/gitlab-ce/issues/46809#note_86604455, it was mentioned multiple times, that screenshots will be in their own zip file,

That internally (for runner) gets converted into the following artifacts definition:

selenium_logs: paths: [selenium/*/selenium-log.json], format: gzip, type: selenium_logs
selenium_screenshots: paths: [selenium/*/*/screenshots/*.png], format: zip, type: selenium_screenshots

https://gitlab.com/gitlab-org/gitlab-ce/issues/46809#note_86811694

Internal API endpoint:

GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/31b6a0fb1403d76c6810811f12ce35a1/screenshots/1612180645-1537364412515.png
GET <namespace>/<project>/-/jobs/<job-iid>/reports/selenium/<another-session-id>/screenshots/<other-screenshot-id>.png

As a note, we can already serve screenshots from the raw artifacts which is currently being used in the PoC with some hard-coded paths, https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/6051

<namespace>/<project>/builds/3201/artifacts/raw/selenium/ba2773ceea765b6e40536a0e4381a31e/screenshots/1612180645-1537364412515.png

cc @ayufan @dosuken123 (separate thoughts welcome)

Edited by Eric Eastwood