Skip to content

Quarantine tests in bulk_user_assignment_spec

Ievgen Chernikovrequested to merge
quar-returns-success-and-failed-assignments into master

What does this MR do and why?

Quarantine the following tests:

  • DuoPro::BulkUserAssignment#execute on self managed instances when the AddOn is purchased with not enough seats returns success and failed assignments and stops execution
  • DuoPro::BulkUserAssignment#execute on Gitlab.com with bulk assignment with not enough seats returns success and failed assignments and stops execution

The tests match one of the following conditions:

  1. has either flakiness1 or flakiness2 label set, which means the number of reported failures is at or above 95 percentile, indicating unusually high failure count.
  2. has severity1 label set, which means the number of reported failures spiked and exceeded its daily threshold.

This MR quarantines the test. This is a discussion starting point to let the responsible group know about the flakiness so that they can take action:

  • accept the merge request and schedule the associated issue to improve the test
  • close the merge request in favor of another merge request to delete the test

Please follow the Flaky tests management process to help us increase master stability.

References

Tests failure issues:

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Merge request reports