2021-02-08 Flaky examples

This report includes several parts:

Top 20 flaky tests sorted by occurrences per file

./spec/support/shared_examples/features/discussion_comments_shared_example.rb: 16
./spec/features/issuables/markdown_references/jira_spec.rb: 15
./spec/support/shared_examples/features/variable_list_shared_examples.rb: 14
./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb: 10
./spec/features/merge_request/user_sees_merge_widget_spec.rb: 10
./spec/features/issues/user_edits_issue_spec.rb: 6
./spec/features/search/user_uses_header_search_field_spec.rb: 4
./spec/features/merge_request/user_resolves_diff_notes_and_discussions_resolve_spec.rb: 4
./spec/features/projects/settings/user_transfers_a_project_spec.rb: 3
./spec/features/issues/user_creates_branch_and_merge_request_spec.rb: 3
./spec/features/triggers_spec.rb: 3
./spec/features/projects/commit/user_comments_on_commit_spec.rb: 3
./spec/features/merge_request/user_posts_notes_spec.rb: 3
./spec/features/issues/create_issue_for_single_discussion_in_merge_request_spec.rb: 3
./spec/support/shared_examples/features/reportable_note_shared_examples.rb: 3
./spec/features/issues/form_spec.rb: 2
./ee/spec/graphql/ee/resolvers/namespace_projects_resolver_spec.rb: 2
./ee/spec/migrations/migrate_scim_identities_to_saml_for_new_users_spec.rb: 2
./spec/features/merge_request/user_sees_cherry_pick_modal_spec.rb: 2
./ee/spec/features/projects/settings/pipeline_subscriptions_spec.rb: 2

Number of flaky tests per type

features: 85
support: 46
spec: 21
models: 7
requests: 6
lib: 5
controllers: 4
migrations: 2

Flaky tests per day for the last 20 days

2021-02-07: 59 files
	["./spec/controllers/concerns/renders_commits_spec.rb", "./spec/controllers/projects/merge_requests_controller_spec.rb", "./spec/features/projects/show/user_manages_notifications_spec.rb", "./spec/models/group_spec.rb", "./spec/models/group_spec.rb", "./ee/spec/support/shared_examples/features/protected_branches_access_control_shared_examples.rb", "./spec/support/shared_examples/features/snippets_shared_examples.rb", "./spec/features/user_can_display_performance_bar_spec.rb", "./ee/spec/services/ci/minutes/batch_reset_service_spec.rb", "./spec/support/shared_examples/features/reportable_note_shared_examples.rb", "./spec/controllers/projects/pipelines_controller_spec.rb", "./spec/features/projects/pipelines/pipelines_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/controllers/projects/merge_requests_controller_spec.rb", "./spec/features/merge_request/user_resolves_diff_notes_and_discussions_resolve_spec.rb", "./spec/features/merge_request/user_resolves_diff_notes_and_discussions_resolve_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./ee/spec/lib/gitlab/graphql/aggregations/vulnerabilities/lazy_user_notes_count_aggregate_spec.rb", "./spec/features/global_search_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./ee/spec/models/project_spec.rb", "./spec/features/projects/commit/user_reverts_commit_spec.rb", "./spec/features/search/user_uses_header_search_field_spec.rb", "./spec/features/search/user_uses_header_search_field_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/issues/user_edits_issue_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./ee/spec/features/projects/settings/pipeline_subscriptions_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/support/shared_examples/features/packages_shared_examples.rb", "./spec/features/issues/user_edits_issue_spec.rb", "./ee/spec/migrations/migrate_scim_identities_to_saml_for_new_users_spec.rb", "./spec/requests/git_http_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/features/user_sees_revert_modal_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/requests/api/merge_requests_spec.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/lib/gitlab/database/count/reltuples_count_strategy_spec.rb", "./ee/spec/features/issues/epic_in_issue_sidebar_spec.rb", "./spec/requests/api/issues/get_project_issues_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/migrations/schedule_link_lfs_objects_projects_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb"])
2021-02-06: 26 files
	["./ee/spec/controllers/registrations/projects_controller_spec.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/support/shared_examples/features/reportable_note_shared_examples.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./spec/lib/atlassian/jira_connect/client_spec.rb", "./spec/features/triggers_spec.rb", "./spec/features/triggers_spec.rb", "./spec/requests/api/merge_requests_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/callouts/registration_enabled_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/features/merge_request/user_reverts_merge_request_spec.rb", "./ee/spec/graphql/ee/resolvers/namespace_projects_resolver_spec.rb", "./spec/features/issues/issue_sidebar_spec.rb", "./spec/features/issues/user_edits_issue_spec.rb", "./ee/spec/features/search/user_searches_for_epics_spec.rb", "./spec/features/projects/commit/cherry_pick_spec.rb", "./spec/features/projects/settings/user_transfers_a_project_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/features/projects/settings/user_transfers_a_project_spec.rb", "./ee/spec/features/epics/shortcuts_epic_spec.rb", "./spec/features/projects/commit/user_comments_on_commit_spec.rb"])
2021-02-05: 21 files
	["./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/models/release_highlight_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/features/issues/user_edits_issue_spec.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/features/triggers_spec.rb", "./spec/requests/api/jobs_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./spec/features/merge_request/user_resolves_diff_notes_and_discussions_resolve_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/features/issues/user_edits_issue_spec.rb", "./spec/models/pages_deployment_spec.rb", "./spec/features/projects/commit/user_comments_on_commit_spec.rb", "./spec/lib/gitlab/instrumentation/redis_spec.rb", "./spec/features/issues/form_spec.rb", "./spec/features/issues/form_spec.rb", "./spec/models/deployment_metrics_spec.rb", "./spec/features/projects/issues/design_management/user_uploads_designs_spec.rb", "./spec/features/projects/settings/user_transfers_a_project_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb"])
2021-02-04: 17 files
	["./ee/spec/migrations/set_resolved_state_on_vulnerabilities_spec.rb", "./ee/spec/services/ci/minutes/batch_reset_service_spec.rb", "./spec/features/merge_request/user_resolves_diff_notes_and_discussions_resolve_spec.rb", "./spec/features/issues/create_issue_for_single_discussion_in_merge_request_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/features/merge_request/user_posts_notes_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/merge_request/user_sees_suggest_pipeline_spec.rb", "./spec/models/custom_emoji_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/features/merge_request/user_posts_notes_spec.rb", "./spec/models/clusters/applications/runner_spec.rb", "./spec/features/merge_request/user_accepts_merge_request_spec.rb", "./spec/lib/gitlab/database/rename_reserved_paths_migration/v1/rename_projects_spec.rb", "./spec/features/issues/user_edits_issue_spec.rb", "./spec/features/issuables/markdown_references/jira_spec.rb"])
2021-02-03: 24 files
	["./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/features/admin/admin_mode/login_spec.rb", "./spec/features/issues/create_issue_for_discussions_in_merge_request_spec.rb", "./spec/features/merge_request/user_assigns_themselves_spec.rb", "./spec/features/merge_request/user_resolves_conflicts_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./ee/spec/controllers/registrations/projects_controller_spec.rb", "./spec/features/issues/user_creates_branch_and_merge_request_spec.rb", "./spec/features/issues/user_creates_branch_and_merge_request_spec.rb", "./spec/features/merge_request/user_sees_cherry_pick_modal_spec.rb", "./spec/features/merge_request/user_sees_avatar_on_diff_notes_spec.rb", "./spec/support/shared_examples/features/reportable_note_shared_examples.rb", "./ee/spec/features/projects/settings/pipeline_subscriptions_spec.rb", "./spec/migrations/schedule_backfill_push_rules_id_in_projects_spec.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./ee/spec/lib/ee/gitlab/cleanup/orphan_job_artifact_files_batch_spec.rb", "./ee/spec/features/projects/custom_projects_template_spec.rb", "./ee/spec/support/shared_examples/quick_actions/issuable/unassign_shared_examples.rb", "./ee/spec/graphql/ee/resolvers/namespace_projects_resolver_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/support/shared_examples/features/project_upload_files_shared_examples.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb"])
2021-02-02: 21 files
	["./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/features/merge_request/batch_comments_spec.rb", "./ee/spec/features/epics/shortcuts_epic_spec.rb", "./spec/features/merge_request/batch_comments_spec.rb", "./spec/features/search/user_uses_header_search_field_spec.rb", "./spec/features/merge_request/user_resolves_conflicts_spec.rb", "./spec/features/issues/create_issue_for_single_discussion_in_merge_request_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/search/user_uses_header_search_field_spec.rb", "./spec/features/projects/commit/user_comments_on_commit_spec.rb", "./spec/features/projects/commit/comments/user_edits_comments_spec.rb", "./spec/features/projects/files/project_owner_creates_license_file_spec.rb", "./spec/features/merge_request/user_sees_cherry_pick_modal_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/features/issues/user_creates_branch_and_merge_request_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./ee/spec/migrations/migrate_scim_identities_to_saml_for_new_users_spec.rb", "./spec/features/issues/create_issue_for_single_discussion_in_merge_request_spec.rb", "./spec/support/shared_examples/features/resolving_discussions_in_issues_shared_examples.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/features/issues/create_issue_for_discussions_in_merge_request_spec.rb"])
2021-02-01: 8 files
	["./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/features/merge_request/user_posts_notes_spec.rb", "./spec/support/shared_examples/features/variable_list_shared_examples.rb", "./spec/requests/api/groups_spec.rb", "./spec/features/merge_request/user_sees_merge_widget_spec.rb", "./spec/support/shared_examples/features/discussion_comments_shared_example.rb", "./spec/lib/gitlab/import_export/group/tree_restorer_spec.rb"])

Top 20 flaky test examples sorted by number of reports

./spec/controllers/concerns/renders_commits_spec.rb[1:2:1]: 2830
./spec/controllers/projects/merge_requests_controller_spec.rb[1:8:2:3:1:1:1]: 1001
./spec/models/group_spec.rb[1:42:3:3:1]: 732
./spec/models/group_spec.rb[1:42:1:1:1]: 732
./spec/features/projects/snippets/show_spec.rb[1:1:1:1]: 331
./spec/features/user_can_display_performance_bar_spec.rb[1:2:1:2:1]: 298
./spec/features/projects/show/user_manages_notifications_spec.rb[1:1]: 153
./spec/controllers/projects/merge_requests_controller_spec.rb[1:21:4:1:1]: 152
./spec/features/issuables/markdown_references/jira_spec.rb[1:1:1:1]: 109
./ee/spec/features/protected_branches_spec.rb[1:2:1:8]: 60
./spec/lib/atlassian/jira_connect/client_spec.rb[1:7:2]: 31
./ee/spec/controllers/registrations/projects_controller_spec.rb[1:2:2:1]: 30
./spec/features/reportable_note/commit_spec.rb[1:1:1:4]: 28
./spec/models/release_highlight_spec.rb[1:1:1:1]: 26
./ee/spec/migrations/set_resolved_state_on_vulnerabilities_spec.rb[1:1:1]: 26
./ee/spec/services/ci/minutes/batch_reset_service_spec.rb[1:1:1:2:1]: 24
./spec/features/merge_request/batch_comments_spec.rb[1:1:9:2]: 21
./spec/controllers/projects/pipelines_controller_spec.rb[1:1:1:2]: 20
./spec/features/merge_request/user_resolves_diff_notes_and_discussions_resolve_spec.rb[1:2:1:5:3:2]: 19
./spec/features/merge_request/user_sees_merge_widget_spec.rb[1:20:3:7:1]: 18

failureflaky-test issues

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information