Skip to content

Identify backend test gaps in merge request pipelines

Context

In &6 (closed), we have identified several test gaps causing broken masters (see &6 (comment 1217676175) and &6 (comment 1221776683)). All of them have been patched.

Goal

This issue is about trying to find test prediction gaps in merge requests, not just ones causing master broken incidents. This is to help prevent master broken incidents ahead of time.

Technical difficulties

In https://app.periscopedata.com/app/gitlab/1116767/Test-Intelligence-Accuracy?widget=16235818&udv=0, we have compiled a list of predictive pipelines (used to be called "minimal pipelines") that passed, and full pipelines that failed afterwards.

A full pipeline might have failed because of an infrastructure issue or a flaky test. Those will need to be filtered out of this list for us to have interesting data to work with.

Edited by David Dieulivol