Create flaky test issues for “rspec in new process” successful retries

Context

In gitlab-org/ruby/gems/gitlab_quality-test_tooling!77 (merged), we introduced a way to create flaky test issues from RSpec reports.

Goal

Use this exe/flaky-test-issue script to create flaky test issues when we detected tests that were successful after a retry in a new rspec process.

Technical considerations

  • Please create a separate GitLab project to report those issues, as there might be a lot of issues coming in from MR pipelines.
  • Ensure that a failure in this script does NOT fail the job, as it is just reporting.
  • Consider reporting errors from this script to Sentry (we might need to create a new project for it)
Edited by David Dieulivol