Multiple E2E failures in main/master due to DB performance issues with new instances
Summary
Repeated failures in master pipelines however cannot be reliably reproduced on local (intermittent), similar failures against GDK appear to resolve after gdk restart
. Trying to find offending commit via git bissect
also proved to be unsuccessful.
First failing pipeline: https://gitlab.com/gitlab-org/gitlab/-/pipelines/806371641
Latest: https://gitlab.com/gitlab-org/gitlab/-/pipelines/806640401
Failing specs:
- ./qa/specs/features/browser_ui/1_manage/user/follow_user_activity_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/suggestions/batch_suggestion_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/suggestions/custom_commit_suggestion_spec.rb
- ./qa/specs/features/browser_ui/4_verify/ci_variable/pipeline_with_protected_variable_spec.rb
- ./qa/specs/features/browser_ui/3_create/web_ide/review_merge_request_spec.rb
- ./qa/specs/features/browser_ui/3_create/web_ide/open_web_ide_from_diff_tab_spec.rb
- ./qa/specs/features/api/1_manage/group_access_token_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/view_merge_request_diff_patch_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/revert/reverting_merge_request_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/squash_merge_request_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/cherry_pick/cherry_pick_commit_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb
- ./qa/specs/features/ee/browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb
- ./qa/specs/features/ee/browser_ui/2_plan/contribution_analytics_spec.rb
- ./qa/specs/features/browser_ui/3_create/merge_request/revert/revert_commit_spec.rb
- ./qa/specs/features/api/3_create/merge_request/push_options_mwps_spec.rb
- ./qa/specs/features/ee/browser_ui/13_secure/enable_scanning_from_configuration_spec.rb
- ./qa/specs/features/ee/browser_ui/3_create/repository/code_owners_with_protected_branch_and_squashed_commits_spec.rb
- ./qa/specs/features/ee/browser_ui/3_create/merge_request/approval_rules_spec.rb
Stack trace (example as it varies from test to test)
Failure/Error: expect(merge_request).to have_file(revertible_merge_request.file_name)
QA::Support::Repeater::WaitExceededError:
Page did not fully load. This could be due to an unending async request or loading icon.
Screenshot / HTML page
Possible fixes
Edited by Dan Davison