Automation for new spec failure issue creation

Objective

To automate creating of issue to capture a new broken spec in master pipeline.

Context

Triage of #broken-master channel involves creating a new issue for each new spec failure discovered. This usually involves the following:

  1. Open the job URL from slack
  2. Look for failed test in rspec logs
  3. Open a new issue form in gitlab-org/gitlab
  4. Copy & paste failed spec file and line number into issue title
  5. Copy & paste job failure trace into issue description
  6. Submit (create) issue
  7. Copy new issue URL to slack

Proposal

  1. Create job artifact to capture failed spec file and the line number. This could be a json of "path_to/abc_spec.rb:123" => "snippets of job failure"
  2. Create post-test job that does the following when there is a failure:
    • check if an issue already exists
    • if none exists, create issue using the above artifact.
  3. Post the issue URL to slack #master-broken

/cc @gl-quality/eng-prod WDYT? Can we leverage anything in the flaky examples report?

Edited by Albert Salim