Make Duo Agentic Chat feature specs more stable
Problem
Duo Agentic Chat feature specs have been consistently flaky in CI, blocking pipelines and affecting developer productivity. Two tracked instances illustrate the problem:
- gitlab-org/quality/test-failure-issues#43218 reports failures in
ee/spec/features/merge_request/user_sees_approval_widget_spec.rb, with a dominant error related to a missing type for an undefinedduo_workflow_incremen...feature. Over 14 days it blocked 28 pipelines and caused 33 job failures. - gitlab-org/quality/test-failure-issues#42787 reports failures in
ee/spec/features/blob_file_name_spec.rb, with a dominant error ofexpected to find text "workflow retried"— a Duo Agentic Chat UI assertion. Over 14 days it blocked 38 pipelines and caused 40 job failures. A draft fix attempted to address this with explicit waits, and the spec was eventually quarantined.
The common thread is that these specs exercise Duo Agentic Chat behavior and are sensitive to timing, UI state, and implementation details of the chat client. This makes them unreliable in CI and costly to maintain.
Proposed Solution
1. Convert Duo Chat feature specs to MSW integration tests
Where possible, Duo Chat feature specs should be converted to Mock Service Worker (MSW) integration tests. MSW integration tests run in a controlled environment without a real browser, eliminating timing and UI-state flakiness that is inherent to feature specs. This reduces the total number of feature specs and moves coverage to a faster, more reliable layer.
2. Avoid embedding shared Duo Chat feature specs across multiple pages unless context meaningfully changes the outcome
Shared Duo Chat feature spec examples (e.g., shared contexts or shared examples) should not be included in multiple page-level specs unless the page context significantly changes the behavior under test. When the chat client behavior is identical regardless of the host page, a single representative spec is sufficient. Duplicating the same shared spec across many pages multiplies flakiness risk without adding meaningful coverage.