Make Duo Agentic Chat feature specs more stable

Problem

Duo Agentic Chat feature specs have been consistently flaky in CI, blocking pipelines and affecting developer productivity. Two tracked instances illustrate the problem:

  • gitlab-org/quality/test-failure-issues#43218 reports failures in ee/spec/features/merge_request/user_sees_approval_widget_spec.rb, with a dominant error related to a missing type for an undefined duo_workflow_incremen... feature. Over 14 days it blocked 28 pipelines and caused 33 job failures.
  • gitlab-org/quality/test-failure-issues#42787 reports failures in ee/spec/features/blob_file_name_spec.rb, with a dominant error of expected to find text "workflow retried" — a Duo Agentic Chat UI assertion. Over 14 days it blocked 38 pipelines and caused 40 job failures. A draft fix attempted to address this with explicit waits, and the spec was eventually quarantined.

The common thread is that these specs exercise Duo Agentic Chat behavior and are sensitive to timing, UI state, and implementation details of the chat client. This makes them unreliable in CI and costly to maintain.

Proposed Solution

1. Convert Duo Chat feature specs to MSW integration tests

Where possible, Duo Chat feature specs should be converted to Mock Service Worker (MSW) integration tests. MSW integration tests run in a controlled environment without a real browser, eliminating timing and UI-state flakiness that is inherent to feature specs. This reduces the total number of feature specs and moves coverage to a faster, more reliable layer.

2. Avoid embedding shared Duo Chat feature specs across multiple pages unless context meaningfully changes the outcome

Shared Duo Chat feature spec examples (e.g., shared contexts or shared examples) should not be included in multiple page-level specs unless the page context significantly changes the behavior under test. When the chat client behavior is identical regardless of the host page, a single representative spec is sufficient. Duplicating the same shared spec across many pages multiplies flakiness risk without adding meaningful coverage.

Edited by Enrique Alcántara