Add the exe/existing-test-health-issue executable

From Draft to Ready

• This MR was tested in Auto-retry jobs when they failed due to a known... (gitlab-org/gitlab!168541 - merged)

Context

Contributes to gitlab-org/gitlab#497138 (closed)

What does this MR do and why?

• Add the TestHealthIssueFinder class, and the exe/existing-test-health-issue executable

How to set up and validate locally

1. checkout this branch
2. Create an RSpec report locally (or download one from a recent job if you'd like 👍 )

Create RSpec report with one failed test in it

cat > ~/Desktop/failed-tests-rspec-report.json <<JSON
{
  "version": "3.12.2",
  "messages": ["Run options: include {:last_run_status=>\"failed\"}"],
  "examples": [
    {
      "id": "./ee/spec/models/gitlab/seat_link_data_spec.rb[1:3:3:1]",
      "description": "includes the add-on metrics in the payload",
      "full_description": "Gitlab::SeatLinkData.to_json when the instance has add-on purchases includes the add-on metrics in the payload",
      "status": "failed",
      "file_path": "./ee/spec/models/gitlab/seat_link_data_spec.rb",
      "line_number": 129,
      "run_time": 2.166395459,
      "pending_message": null,
      "testcase": null,
      "quarantine": null,
      "screenshot": null,
      "product_group": null,
      "feature_category": null,
      "ci_job_url": "https://gitlab.com/gitlab-org/gitlab/-/jobs/8000569154",
      "retry_attempts": 0,
      "level": "unit",
      "allowed_to_be_slow": null
    }
  ],
  "summary": {
    "duration": 43.646335449,
    "example_count": 1,
    "failure_count": 1,
    "pending_count": 0,
    "errors_outside_of_examples_count": 0
  },
  "summary_line": "1 example, 1 failure"
}
JSON

3. Try to find test-health:failures for the spec above:

# Should output an issue URL (https://gitlab.com/gitlab-org/gitlab/-/issues/497876), and exit with an exit code 0.
exe/existing-test-health-issue \
  -i ~/Desktop/failed-tests-rspec-report.json \
  -p 'gitlab-org/gitlab' \
  -t "${GITLAB_API_PRIVATE_TOKEN}" \
  --health-problem-type failures

4. Try to find test-health:slow for the spec above:

# Will give you nothing, and exit status code of 1.
exe/existing-test-health-issue \
  -i ~/Desktop/failed-tests-rspec-report.json \
  -p 'gitlab-org/gitlab' \
  -t "${GITLAB_API_PRIVATE_TOKEN}" \
  --health-problem-type slow

5. Try to find test-health:pass-after-retry for the spec above:

# Will give you nothing, and exit status code of 1.
exe/existing-test-health-issue \
  -i ~/Desktop/failed-tests-rspec-report.json \
  -p 'gitlab-org/gitlab' \
  -t "${GITLAB_API_PRIVATE_TOKEN}" \
  --health-problem-type pass-after-retry

Bonus: If you change the status in the RSpec report from failed to passed, and run the command with --health-problem-type pass-after-retry, you'll get issue gitlab-org/gitlab#497876 (closed) back 🎉 .

This is because for test-health:pass-after-retry , we only check tests that passed. For test-health:failures , we only take jobs that failed.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.