:waves: As part of &22879+, I've been finding digging up a lot of information on flakes; especially "quiet" ones that get pass on automatic retry. They don't fail a pipeline and so they can go under the radar, but they lead to longer overall pipeline times, and sometimes _do_ fail pipelines when they fail on retry. In the three months 2026-05-24 – 2026-08-24, `spec/features/work_items/detail/shortcuts_work_item_spec.rb` has had 20,167 pass-on-retry flakes, nearly 99% of which are `FrozenError`s spread throughout every example in the file, affecting 6% of runs. This has likely stemmed from changes made as part of https://gitlab.com/gitlab-org/gitlab/-/work_items/600267+, which means we get `FrozenError`s instead of subtler failures that look like logic bugs. ## Related issues - gitlab-org/quality/test-failure-issues#43901+ - gitlab-org/quality/test-failure-issues#39770+ - gitlab-org/quality/test-failure-issues#35811+ - gitlab-org/quality/test-failure-issues#39448+ ## Active flaking examples (with last flake on/after 2026-08-03) Below is data collated by LLM. Note there's a very small number of non-`FrozenError` exceptions seen too (the other 1%). ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:3:1]` - name: Work item keyboard shortcuts for signed in user navigation pressing . opens web IDE - flake events: 12318 (3207 on master pipelines); active 2026-05-25 → 2026-08-17 (13 weeks) - recent weekly: 07-13: 1611, 07-20: 542, 07-27: 1810, 08-03: 1480, 08-10: 1017, 08-17: 58 - exceptions: `['FrozenError']`; `['Net::ReadTimeout']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `can't modify frozen attributes` ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:2:3:1]` - name: Work item keyboard shortcuts for signed in user pressing r when replying to a comment quotes the selected text in the discussion reply form - flake events: 6137 (1722 on master pipelines); active 2026-05-25 → 2026-08-17 (13 weeks) - recent weekly: 07-13: 848, 07-20: 342, 07-27: 1085, 08-03: 787, 08-10: 650, 08-17: 35 - exceptions: `['FrozenError']`; `['Net::ReadTimeout']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `can't modify frozen attributes` ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:2:2]` - name: Work item keyboard shortcuts for signed in user pressing r quotes the selected text in the discussion reply form - flake events: 1317 (467 on master pipelines); active 2026-05-25 → 2026-08-17 (13 weeks) - recent weekly: 07-13: 184, 07-20: 97, 07-27: 260, 08-03: 155, 08-10: 162, 08-17: 5 - exceptions: `['FrozenError']`; `['Net::ReadTimeout']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `can't modify frozen attributes` ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:2:1]` - name: Work item keyboard shortcuts for signed in user pressing r focuses main comment field by default - flake events: 173 (61 on master pipelines); active 2026-05-25 → 2026-08-17 (13 weeks) - recent weekly: 07-13: 15, 07-20: 10, 07-27: 21, 08-03: 21, 08-10: 17, 08-17: 46 - exceptions: `['FrozenError']`; `['RSpec::Expectations::ExpectationNotMetError','BrowserConsoleHelpers::BrowserConsoleError']`; `['RSpec::Expectations::ExpectationNotMetError']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `can't modify frozen attributes` - message sample: `expected to find css ".js-main-target-form .js-gfm-input:focus" but there were no matches Timeout (30s) reached while running a waiting Capybara finder.` ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:1:1]` - name: Work item keyboard shortcuts for signed in user sidebar pressing m opens milestones dropdown for editing - flake events: 68 (20 on master pipelines); active 2026-06-29 → 2026-08-17 (4 weeks) - recent weekly: 06-29: 1, 08-03: 2, 08-10: 1, 08-17: 64 - exceptions: `['RSpec::Expectations::ExpectationNotMetError','BrowserConsoleHelpers::BrowserConsoleError']`; `['RSpec::Expectations::ExpectationNotMetError']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `expected to find css ".gl-new-dropdown-panel" within #<Capybara::Node::Element tag="section" path="/HTML/BODY[1]/DIV[2]/DIV[3]/DIV[1]/DIV[1]/DIV[2]/DIV[1]/DIV[2]/MAIN[1]/DIV[4]/DIV[1]/DIV[2]/DIV[2]/DIV[1]/DIV[2]/SECTION[1]/DIV[1]/SECTION[3]"> but there were no matches Timeout (30s) reached while ru` - message sample: `expected to find css ".gl-new-dropdown-panel" within #<Capybara::Node::Element tag="section" path="/HTML/BODY[1]/DIV[2]/DIV[3]/DIV[1]/DIV[1]/DIV[2]/DIV[1]/DIV[2]/MAIN[1]/DIV[5]/DIV[1]/DIV[2]/DIV[2]/DIV[1]/DIV[2]/SECTION[1]/DIV[1]/SECTION[3]"> but there were no matches Timeout (30s) reached while ru` ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:1:2]` - name: Work item keyboard shortcuts for signed in user sidebar pressing l opens labels dropdown for editing - flake events: 55 (10 on master pipelines); active 2026-07-06 → 2026-08-17 (6 weeks) - recent weekly: 07-06: 2, 07-13: 2, 07-27: 2, 08-03: 1, 08-10: 4, 08-17: 44 - exceptions: `['RSpec::Expectations::ExpectationNotMetError','BrowserConsoleHelpers::BrowserConsoleError']`; `['RSpec::Expectations::ExpectationNotMetError']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `expected to find css ".gl-new-dropdown-panel" within #<Capybara::Node::Element tag="section" path="/HTML/BODY[1]/DIV[2]/DIV[3]/DIV[1]/DIV[1]/DIV[2]/DIV[1]/DIV[2]/MAIN[1]/DIV[5]/DIV[1]/DIV[2]/DIV[2]/DIV[1]/DIV[2]/SECTION[1]/DIV[1]/SECTION[2]"> but there were no matches Timeout (30s) reached while ru` - message sample: `tab crashed (Session info: chrome=138.0.7204.183)` ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:1:3]` - name: Work item keyboard shortcuts for signed in user sidebar pressing a opens assignee dropdown for editing - flake events: 50 (12 on master pipelines); active 2026-07-06 → 2026-08-24 (5 weeks) - recent weekly: 07-06: 1, 08-03: 3, 08-10: 2, 08-17: 43, 08-24: 1 - exceptions: `['RSpec::Expectations::ExpectationNotMetError','BrowserConsoleHelpers::BrowserConsoleError']`; `['RSpec::Expectations::ExpectationNotMetError']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `expected to find css ".gl-new-dropdown-panel" within #<Capybara::Node::Element tag="section" path="/HTML/BODY[1]/DIV[2]/DIV[3]/DIV[1]/DIV[1]/DIV[2]/DIV[1]/DIV[2]/MAIN[1]/DIV[5]/DIV[1]/DIV[2]/DIV[2]/DIV[1]/DIV[2]/SECTION[1]/DIV[1]/SECTION[1]"> but there were no matches Timeout (30s) reached while ru` ### `spec/features/work_items/detail/shortcuts_work_item_spec.rb[1:1:1:4]` - name: Work item keyboard shortcuts for signed in user sidebar pressing e starts editing mode - flake events: 49 (9 on master pipelines); active 2026-07-06 → 2026-08-17 (4 weeks) - recent weekly: 07-06: 2, 08-03: 2, 08-10: 2, 08-17: 43 - exceptions: `['FrozenError']`; `['RSpec::Expectations::ExpectationNotMetError','BrowserConsoleHelpers::BrowserConsoleError']`; `['RSpec::Expectations::ExpectationNotMetError']`; `['Selenium::WebDriver::Error::WebDriverError']` - message sample: `can't modify frozen attributes` - message sample: `expected to find css "[data-testid=\"work-item-title-input\"]" but there were no matches Timeout (30s) reached while running a waiting Capybara finder. Consider using a non-waiting finder. See https://www.cloudbees.com/blog/faster-rails-tests`