Revisiting the Quarantining Process for E2E Tests

Looks like there are few gaps in the Quarantining Process that we follow, as a result of which, we tend to keep quarantining the tests and don't have a defined process to fix and de-quarantine the same.

Please find below the proposal to solve this issue.

Proposal

Existing Quarantined Tests

  • I’m planning to create an issue which has a checklist of all the quarantined test’s issues. (Similar to the unlabelled issues list for triaging) to fix the existing quarantined tests.
  • This list will be shared by the TAEs in Dev team (~17 tests)

Change in Quarantining/De-quarantining Process

  • We should have a triaging package which generates a new issue every monday which has the list of open quarantined test issues.
  • This issue should be assigned to the TAE in charge of “Failure Management Rotation” and it should be prioritized above any other tasks in his plate.
  • Within that week, the TAE is expected to debug, fix and dequarantine the test.
  • Whichever tests are quarantined in the current week will again be looked into by the next TAE in rotation and the process repeats.

@gl-quality Please share your thoughts.

Status

Quarantined Tests:

Edited by Ramya Authappan