Flaky test: spec/features/merge_request/user_merges_when_pipeline_succeeds_spec.rb
Summary
Several test examples in spec/features/merge_request/user_merges_when_pipeline_succeeds_spec.rb are flaky, see incidents:
-
gitlab-org/quality/engineering-productivity/master-broken-incidents#1601 (closed) failed on
spec/features/merge_request/user_merges_when_pipeline_succeeds_spec.rb[1:2:1:3:1:1] -
gitlab-org/quality/engineering-productivity/master-broken-incidents#1607 (closed) failed on
spec/features/merge_request/user_merges_when_pipeline_succeeds_spec.rb[1:2:1:3:1:1] -
gitlab-org/quality/engineering-productivity/master-broken-incidents#1603 (closed) failed on
spec/features/merge_request/user_merges_when_pipeline_succeeds_spec.rb[1:1:1:4:1:1] -
gitlab-org/quality/engineering-productivity/master-broken-incidents#1606 (closed) failed on
spec/features/merge_request/user_merges_when_pipeline_succeeds_spec.rb[1:1:1:3:1:1] -
gitlab-org/quality/engineering-productivity/master-broken-incidents#1612 (closed) failed on
spec/features/merge_request/user_merges_when_pipeline_succeeds_spec.rb[1:2:1:2:1:1]
Steps to reproduce
Locally I was not able to reliably reproduce the failures with rspec and rspec bisect.
Please refer to Flaky tests documentation to learn more about how to reproduce them.
Proposed Resolution
In addition to addressing the flakiness, I suggest we simplify the test cases by reducing the length of each spec description. These test failures couldn't produce any screenshot because as seen in the job log:
WARN: HTML source could not be saved. An exception is raised: #<Errno::ENAMETOOLONG: File name too long @ rb_sysopen - /builds/gitlab-org/gitlab/tmp/capybara/merge_request_user_merges_when_pipeline_succeeds_when_there_is_active_pipeline_for_merge_request_enabling_merge_when_pipeline_succeeds_when_it_was_enabled_and_then_canceled_behaves_like_merge_when_pipeline_succeeds_activator_activates_the_merge_when_pipeli.html>. WARN: Screenshot could not be saved. An exception is raised: #<Errno::ENAMETOOLONG: File name too long @ rb_sysopen - /builds/gitlab-org/gitlab/tmp/capybara/merge_request_user_merges_when_pipeline_succeeds_when_there_is_active_pipeline_for_merge_request_enabling_merge_when_pipeline_succeeds_when_it_was_enabled_and_then_canceled_behaves_like_merge_when_pipeline_succeeds_activator_activates_the_merge_when_pipeli.png>.
Please refer to the Resolution guidance to learn more about resolution of broken master.
Once the flaky failure has been fixed on the default branch, open merge requests to cherry-pick the fix to the active stable branches.