### Problem Statement Flaky test detection only works for RSpec today. Jest, Go, and Rust tests have no way to participate, even though flakiness is just as real a problem for them. The reason is technical: flaky detection needs a same-run retry-confirmation signal, meaning a test failed, was retried in the same run, and failed again. Only RSpec produces that signal today. Jest and Go already export test results, but their retry mechanism isn't wired up yet. Rust (Orbit) doesn't export test results to ClickHouse at all. | Framework | Retry mechanism | Retry-confirmation wired? | Exporter exists? | |---|---|---|---| | RSpec | rerun-failed-in-new-process | Yes | Yes | | Jest | none (job-level CI retry only) | No | Yes | | Go (workhorse) | none | No | Yes | | Rust (Orbit) | `cargo nextest` native retries (3x, exponential backoff) | N/A | No formatter yet | ### Participants - @ddieulivol (DRI) ### Exit Criteria - Jest, Go, and Rust test failures can be safely and deliberately admitted into flaky test detection, one framework at a time, via an audited allowlist (no accidental floods of new data into existing dashboards/alerts). - Jest and Go have a real retry-confirmation mechanism, giving flaky detection the same granularity it has for RSpec. - Rust (Orbit) exports test data to ClickHouse for the first time, with `cargo nextest`'s native retry data captured. - ci-alerts' exception-grouping and failure-category logic correctly handles failures from the new frameworks, without misclassifying or double-alerting. ### Non-goals - Migrating Jest off `jest-jasmine2` to `jest-circus`. The retry pass will be a second CI-script invocation (`jest --onlyFailures`), not `jest.retryTimes()`. ### Issues | Item | Tracking | Status | |------|----------|--------| | Gate non-RSpec frameworks into flaky test detection behind an explicit run_type allowlist (blocking) | https://gitlab.com/gitlab-org/quality/engineering-productivity-infrastructure/-/work_items/145 | Planned | | Wire real testRetried predicate for Jest formatter | https://gitlab.com/gitlab-org/quality/analytics/test-metrics-exporters/-/work_items/1 | Planned | | Add "rerun failed Jest tests only" CI pass in gitlab-org/gitlab | https://gitlab.com/gitlab-org/gitlab/-/work_items/605130 | Planned | | Wire real retry predicate for Go formatter | https://gitlab.com/gitlab-org/quality/analytics/test-metrics-exporters/-/work_items/2 | Planned | | Add "rerun failed Go tests only" CI pass in gitlab-org/gitlab | https://gitlab.com/gitlab-org/gitlab/-/work_items/605131 | Planned | | Build new rust/ formatter package in test-metrics-exporters | https://gitlab.com/gitlab-org/quality/analytics/test-metrics-exporters/-/work_items/3 | Planned | | Integrate rust/ formatter into Orbit CI | https://gitlab.com/gitlab-org/rust/knowledge-graph/-/work_items/326 | Planned | | Revisit ci-alerts exception grouping / failure category handling for new frameworks | https://gitlab.com/gitlab-org/quality/analytics/ci-alerts/-/work_items/423 | Planned | ### Development Log <!-- STATUS NOTE START --> <!-- STATUS NOTE END --> ### Related - Discussion: https://gitlab.com/gitlab-org/quality/analytics/team/-/work_items/704#note_3532354041