Skip to content

Identify top smoke failures using delivery blocker issues and Delivery Deployment Blockers dashboard

As part of Reduce deployment blocked hours due to test fai... (gitlab-org/quality&206), we need to understand the scope of the work by first analyzing data coming in from Delivery. For now, we are focusing on tests in the :smoke suite because they are more directly related to a halted delivery.

In this issue, we can start by looking into data provided in Deployment Blockers issues as well as delivery: Deployment Blockers dashboard. Using these data points (tentatively let's try with 01-07-2024 to 31-07-2025 period then we can adjust accordingly), can we answer these questions:

  1. What are the tests that have failed most often recently?
  2. The root cause(s) of each failure. What are their nature/types?
  3. When do they often occur?
  4. How many retries they took to pass again? Is there any grace period between each retry?
Edited by Tiffany Rea