Flaky verify spec (!228) · Merge requests · GitLab.org / Quality Department / Engineering Productivity / Fast Quarantine

Fast-quarantine process

Edit the rspec/fast_quarantine-gitlab.txt file and add a test entity identifier, in one of the following format:
- An example id, e.g. spec/tasks/gitlab/usage_data_rake_spec.rb[1:5:2:1]
- A test file with a line number, e.g. ee/spec/features/boards/swimlanes/epics_swimlanes_sidebar_spec.rb:42
- A test file, e.g. ee/spec/features/boards/swimlanes/epics_swimlanes_sidebar_spec.rb Note that end-to-end tests can also be fast-quarantined, but you should omit the top-level qa/ folder. For instance, to fast-quarantine qa/qa/specs/features/browser_ui/3_create/repository/add_new_branch_rule_spec.rb, you would add qa/specs/features/browser_ui/3_create/repository/add_new_branch_rule_spec.rb to the rspec/fast_quarantine-gitlab.txt file.
Commit the change to a new branch and open a merge request with a description on why the test is fast-quarantined, with a link to the flaky test issue.
You can merge the merge request yourself or ask for review/approval/merge, as you see fit.
To immediately unblock a pipeline, first rerun the retrieve-test-metadata job (download-fast-quarantine-report in e2e pipelines), and then retry the failed job.
Create a long-term quarantine merge request for the flaky test you just fast-quarantined.
- This is important, as we want to ensure the flaky test was correctly fixed by running tests from the CI/CD pipelines. This cannot be done from this project.
Once the flaky test was long-term quarantined, remove it from the fast-quarantine list.