Test report API is not consistent with the UI when the JUnit report artifacts expired

Summary

One of our customers reported (internal only) that the test report API is not consistent with the UI.

I was able to reproduce the issue on my test instance. Based on my test, this happens when the JUnit artifact has expired.

Steps to reproduce

  1. Create a new project with unit test reports configured.
  2. Trigger a new pipeline and wait for it to complete.
  3. Access the test report using the API to verify that it's showing the complete information.
  4. Manually delete the job artifact by forcing it to expire.
  5. Access the test report using the API again. Notice that it will have the output of:
{"total_time":0,"total_count":0,"success_count":0,"failed_count":0,"skipped_count":0,"error_count":0,"test_suites":[]}

Example Project

I think this is also happening on GitLab.com. Using one of our pipeline as an example: https://gitlab.com/gitlab-org/gitlab-runner/-/pipelines/221552771/test_report

If you try to access the test report via the API, it will return the output above.

What is the current bug behavior?

Test report API will return an all zero output.

What is the expected correct behavior?

It should be consistent with the UI.

Output of checks

This happens on GitLab.com

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

Ensure that there is a clear indication that the artifact is expired. Some suggested proposals:

@drew has confirmed that the API uses Pipeline#test_reports which goes directly to the job artifact records.

The better solution might be to use BuildReportResult in the API and return the counts without the (now expired) test data.

To fix this in the UI, we have this issue: #296963 (closed).

Edited by Gina Doyle