Skip to content

Create flaky test issues for successful test in RSpec JSON report

David Dieulivol requested to merge 297-create_flaky_test_issues into main

Context

Closes #29 (closed)

What does this MR do and why?

Adds an exe/flaky-test-issue script to create flaky tests issues based on a JSON RSpec report file.

How to set up and validate locally

Command

# Note: 51263763 is https://gitlab.com/ddieulivol/sandbox, a test project
$ exe/flaky-test-issues \
  --input-files spec/gitlab_quality/test_tooling/report/fixtures/rspec-reports/flaky/rspec-retry-5258318483.json \
  --project 51263763 \
  --token "${GITLAB_API_PRIVATE_TOKEN}"

Output

Reporting flaky tests in `spec/gitlab_quality/test_tooling/report/fixtures/rspec-reports/flaky/rspec-retry-5258318483.json` as issues in project `51263763` via the API at `https://gitlab.com/api/v4`.
=> Reporting 2 tests in spec/gitlab_quality/test_tooling/report/fixtures/rspec-reports/flaky/rspec-retry-5258318483.json
   => Finding existing issues for flaky test 'Gitlab::Geo.current_node returns a GeoNode instance' (run time: 0.11 seconds)...
   => Existing issue link https://gitlab.com/ddieulivol/sandbox/-/issues/39.
   => Adding the flaky test to the existing issue...

You can see the list of issues in https://gitlab.com/ddieulivol/sandbox/-/issues/?sort=created_date&state=opened. You can also try to close/delete issues to test it some more. Below is a screenshot of one of the issues used for testing:

Screenshot_2023-11-23_at_14.31.37

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by David Dieulivol

Merge request reports