Reliable e2e test report

• Candidates for promotion to blocking (2024-07-22 - 2024-08-05)
  • Ai_powered (1)
  • Analytics (1)
  • Data_stores (4)
  • Create (14)
  • Fulfillment (3)
  • Monitor (5)
  • Govern (13)
  • Manage (18)
  • Package (5)
  • Plan (6)
  • Release (1)
  • Systems (6)
  • Verify (13)
• Blocking specs with failures (2024-07-22 - 2024-08-05)
  • Data_stores (1)
  • Package (3)

Candidates for promotion to blocking (2024-07-22 - 2024-08-05)

Note: MRs will be auto-created for promoting the top 10 specs sorted by most number of successful runs

Total amount: 90

STAGE	COUNT
ai_powered	1
analytics	1
data_stores	4
create	14
fulfillment	3
monitor	5
govern	13
manage	18
package	5
plan	6
release	1
systems	6
verify	13

Ai_powered (1)

Executions table groupduo chat (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Ai-powered Duo Chat when initiating Duo Chat on Self-managed behaves like Duo Chat gets a response back from Duo Chat File: duo_chat_spec.rb	164	0	0%

Analytics (1)

Executions table groupanalytics instrumentation (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Analytics Service ping disabled when disabled from gitlab.yml config has service ping toggle is disabled File: service_ping_disabled_spec.rb	168	0	0%

Data_stores (4)

Executions table grouptenant scale (4)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Data Stores Group with members can be shared with another group with correct access level File: share_group_with_group_spec.rb	371	0	0%
Name: Data Stores prevent forking outside group when disabled allows forking outside of group File: prevent_forking_outside_group_spec.rb	371	0	0%
Name: Data Stores prevent forking outside group when enabled does not allow forking outside of group File: prevent_forking_outside_group_spec.rb	371	0	0%
Name: Configure Mattermost support user creates a group with a mattermost team File: create_group_with_mattermost_team_spec.rb	168	0	0%

Create (14)

Executions table groupremote development (5)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with no Duo Pro add-on behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb	168	0	0%
Name: Create Code Suggestions in Web IDE on Self-managed with no license behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb	168	0	0%
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like a code completion suggestion returns a code completion suggestion which can be accepted File: code_suggestions_in_web_ide_spec.rb	164	0	0%
Name: Create Duo Chat in Web IDE on Self-managed behaves like Duo Chat gets a response back from Duo Chat File: duo_chat_in_web_ide_spec.rb	164	0	0%
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with a Duo Pro add-on when seat is not assigned behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb	162	0	0%

Executions table groupcode creation (4)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Create Code Suggestions when code generation is requested on Self-managed with a valid license with no Duo Pro add-on behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb	168	0	0%
Name: Create Code Suggestions when code generation is requested on Self-managed with no license behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb	168	0	0%
Name: Create Code Suggestions when code generation is requested when streaming on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like code suggestions API using streaming streams a suggestion File: code_suggestions_spec.rb	164	0	0%
Name: Create Code Suggestions when code generation is requested on Self-managed with a valid license with a Duo Pro add-on when seat is not assigned behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb	162	0	0%

Executions table groupcode review (4)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Create Merge Requests with a malformed URL returns 400 File: view_merge_requests_spec.rb	167	0	0%
Name: Create Merge request when merging once behaves like set to auto-merge merges after pipeline succeeds File: merge_request_set_to_auto_merge_spec.rb	166	0	0%
Name: Create Merge request when merging several times behaves like set to auto-merge merges after pipeline succeeds File: merge_request_set_to_auto_merge_spec.rb	148	0	0%
Name: Create Merge request push options merges when pipeline succeeds File: push_options_mwps_spec.rb	93	0	0%

Executions table groupsource code (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Create Push rules using non signed commits restricts files by name and size File: push_rules_spec.rb	874	0	0%

Fulfillment (3)

Executions table groupprovision (2)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Fulfillment Cloud activation code activates instance with correct subscription details File: cloud_activation_spec.rb	168	0	0%
Name: Fulfillment Remove cloud subscription successfully removes a cloud activation and shows flash notice File: cloud_activation_spec.rb	168	0	0%

Executions table grouputilization (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Fulfillment Utilization User Registration when adding and removing a group member consumes a seat on the license File: user_registration_billing_spec.rb	371	0	0%

Monitor (5)

Executions table groupobservability (2)

NAME	RUNS	FAILURES	FAILURE RATE
Name: GitLab Metrics Sidekiq metrics via dedicated server returns 200 OK and serves metrics File: metrics_spec.rb	167	0	0%
Name: GitLab Metrics Web metrics via dedicated server returns 200 OK and serves metrics File: metrics_spec.rb	167	0	0%

Executions table grouprespond (3)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Monitor Recovery alert when using Prometheus integration behaves like triggers recovery alert only resolves the correct alert File: recovery_alert_resolves_correct_alert_spec.rb	168	0	0%
Name: Monitor Recovery alert when using Prometheus integration behaves like triggers recovery alert only closes the correct incident File: recovery_alert_closes_correct_incident_spec.rb	164	0	0%
Name: Monitor Recovery alert when using HTTP endpoint integration behaves like triggers recovery alert only closes the correct incident File: recovery_alert_closes_correct_incident_spec.rb	160	0	0%

Govern (13)

Executions table groupauthentication (9)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Govern Group SAML SSO - Enforced SSO user clones and pushes to project within a group using Git HTTP File: group_saml_enforced_sso_git_access_spec.rb	168	0	0%
Name: Govern Group SAML SSO - Enforced SSO with Snowplow tracking disabled behaves like group membership actions creates a new account automatically and allows to leave group and join again File: group_saml_enforced_sso_new_account_spec.rb	168	0	0%
Name: Govern Group SAML SSO - Enforced SSO with Snowplow tracking enabled behaves like group membership actions creates a new account automatically and allows to leave group and join again File: group_saml_enforced_sso_new_account_spec.rb	168	0	0%
Name: Govern Group SAML SSO - Non enforced SSO lets group admin test settings File: group_saml_non_enforced_sso_spec.rb	168	0	0%
Name: Govern Group SAML SSO - Non enforced SSO user logs in to group with SAML SSO File: group_saml_non_enforced_sso_spec.rb	168	0	0%
Name: Govern Group SAML SSO - Non enforced SSO when SAML SSO is configured with a default membership role adds the new member with access level as set in SAML SSO configuration File: group_saml_non_enforced_sso_spec.rb	168	0	0%
Name: Govern Mattermost login user logs into Mattermost using GitLab OAuth File: log_into_mattermost_via_gitlab_spec.rb	168	0	0%
Name: Govern OAuth connects and logs in with GitHub OAuth File: oauth_login_with_github_spec.rb	168	0	0%
Name: Govern Instance wide SAML SSO user logs in to gitlab with SAML SSO File: login_via_instance_wide_saml_sso_spec.rb	166	0	0%

Executions table groupcompliance (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Govern Merge request approval with SAML SSO can be approved when requiring a password File: saml_sso_merge_request_approve_spec.rb	168	0	0%

Executions table groupthreat insights (3)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Govern Security Reports displays security reports in the group security dashboard File: security_reports_spec.rb	378	0	0%
Name: Govern Vulnerability report in a project can export vulnerability report to csv File: export_vulnerability_report_spec.rb	371	0	0%
Name: Govern vulnerability report with jira integration can successfully create a JIRA issue from vulnerability details page File: vulnerabilities_jira_integration_spec.rb	167	0	0%

Manage (18)

Executions table groupimport and integrate (18)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Manage Jenkins integration integrates and displays build status for MR pipeline in GitLab File: jenkins_build_status_spec.rb	370	0	0%
Name: Manage WebHooks integration sends a push event File: webhook_events_spec.rb	170	0	0%
Name: Manage WebHooks integration sends a tag event File: webhook_events_spec.rb	170	0	0%
Name: Manage WebHooks integration sends a wiki page event File: webhook_events_spec.rb	170	0	0%
Name: Manage WebHooks integration sends an issues and note event File: webhook_events_spec.rb	170	0	0%
Name: Manage GitHub import when imported via UI imports a project File: import_github_repo_spec.rb	168	0	0%
Name: Manage GitHub import when imported via api imports project File: import_github_repo_spec.rb Exceptions: - Fabrication of QA::Resource::ProjectImportedFromGithub using the API failed (500) with '{"message":"500 Internal Server Error"}'.Correlation Id: 01J48DJHXH86C1Y8K0N4MFY6WK	168	0	0%
Name: Manage GitHub import when imported via api imports repo push rules File: import_github_repo_spec.rb Exceptions: - Fabrication of QA::Resource::ProjectImportedFromGithub using the API failed (500) with '{"message":"500 Internal Server Error"}'.Correlation Id: 01J48DJENTPBE5YQXWR7AZ4GNX	168	0	0%
Name: Manage Gitlab migration with EE features imports group epics and iterations File: gitlab_migration_group_spec.rb	168	0	0%
Name: Manage Group WebHooks integration sends subgroup events File: group_webhook_events_spec.rb	168	0	0%
Name: Manage Pipeline status emails when pipeline fails sends an email File: pipeline_status_emails_spec.rb	168	0	0%
Name: Manage Pipeline status emails when pipeline passes sends an email File: pipeline_status_emails_spec.rb	168	0	0%
Name: Manage WebHooks integration sends a merge request event File: webhook_events_spec.rb	168	0	0%
Name: Manage Jira integration closes an issue via a merge request File: jira_basic_integration_spec.rb	167	0	0%
Name: Manage Jira integration closes an issue via pushing a commit File: jira_basic_integration_spec.rb	167	0	0%
Name: Manage Jira issues integration searching issues returns results File: jira_issues_list_spec.rb	167	0	0%
Name: Manage Jira issues integration shows open issues File: jira_issues_list_spec.rb	167	0	0%
Name: Manage Jira issues integration views an issue File: jira_issues_list_spec.rb	167	0	0%

Package (5)

Executions table groupcontainer registry (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Package Dependency Proxy using docker:24.0.1 and a ci job token pulls an image using the dependency proxy File: dependency_proxy_spec.rb	177	0	0%

Executions table grouppackage registry (4)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Package Maven group level endpoint duplication setting when enabled allows users to publish duplicates File: maven_group_level_spec.rb	406	0	0%
Name: Package Maven group level endpoint via maven using a ci job token pushes and pulls a maven package File: maven_group_level_spec.rb	406	0	0%
Name: Package Maven group level endpoint via maven using a personal access token pushes and pulls a maven package File: maven_group_level_spec.rb	406	0	0%
Name: Package Maven group level endpoint via maven using a project deploy token pushes and pulls a maven package File: maven_group_level_spec.rb	406	0	0%

Plan (6)

Executions table groupproduct planning (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Plan Epics roadmap presents epic on roadmap File: roadmap_spec.rb	168	0	0%

Executions table groupknowledge (5)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Plan Testing project wiki can delete a page with spaces in the path File: project_based_page_deletion_spec.rb	379	0	0%
Name: Plan Testing wiki content creation inside a project by adding a home page to the wiki using git push File: project_based_content_creation_spec.rb	379	0	0%
Name: Plan Testing wiki content creation inside a project by adding a second page to the wiki using git push File: project_based_content_creation_spec.rb	379	0	0%
Name: Plan Testing wiki content manipulation inside a project by manipulating content on the page using git push File: project_based_content_manipulation_spec.rb	378	0	0%
Name: Plan Testing wiki content manipulation inside a project by manipulating content on the page with spaces File: project_based_content_manipulation_spec.rb	378	0	0%

Release (1)

Executions table groupenvironments (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Release Operations Dashboard has many pipelines with appropriate statuses File: pipeline_status_on_operation_dashboard_spec.rb	166	0	0%

Systems (6)

Executions table groupgitaly (6)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Systems Gitaly Cluster replication queue allows replication of different repository after interruption File: praefect_replication_queue_spec.rb	168	0	0%
Name: Systems Gitaly backend node recovery recovers from dataloss File: backend_node_recovery_spec.rb	168	0	0%
Name: Systems Gitaly distributed reads reads from each node File: distributed_reads_spec.rb	168	0	0%
Name: Systems Gitaly distributed reads when a node is unhealthy does not read from the unhealthy node File: distributed_reads_spec.rb	168	0	0%
Name: Systems with Gitaly automatic failover and recovery automatically fails over File: automatic_failover_and_recovery_spec.rb	168	0	0%
Name: Systems with Gitaly automatic failover and recovery when recovering from dataloss after failover automatically reconciles File: automatic_failover_and_recovery_spec.rb	168	0	0%

Verify (13)

Executions table grouppipeline authoring (6)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Verify Pipeline with image:pull_policy when policy is allowed with [always] policy applies pull policy in job correctly File: pipeline_with_image_pull_policy_spec.rb	377	0	0%
Name: Verify Pipeline with image:pull_policy when policy is allowed with [never] policy applies pull policy in job correctly File: pipeline_with_image_pull_policy_spec.rb	377	0	0%
Name: Verify CI catalog when sorting with created at behaves like ascending order displays from first to last File: ci_catalog_sorting_spec.rb	370	0	0%
Name: Verify CI catalog when sorting with created at behaves like descending order by default displays from last to first File: ci_catalog_sorting_spec.rb	370	0	0%
Name: Verify CI component runs in project pipeline with correct inputs File: run_component_in_project_pipeline_spec.rb	367	0	0%
Name: Verify CI catalog when sorting with released at behaves like descending order by default displays from last to first File: ci_catalog_sorting_spec.rb	167	0	0%

Executions table grouppipeline execution (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Verify when pipeline is blocked can still merge MR successfully File: merge_mr_when_pipline_is_blocked_spec.rb	166	0	0%

Executions table grouppipeline security (6)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest child pipeline failed unlocks job artifacts from previous successful pipeline family because the latest parent is successful File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb	167	0	0%
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest pipeline family is successful unlocks job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb	167	0	0%
Name: Verify Unlocking job artifacts across parent-child pipelines with strategy:depend when latest pipeline family is successful unlocks job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb	166	0	0%
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest parent pipeline failed does not unlock job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb	166	0	0%
Name: Verify Unlocking job artifacts across parent-child pipelines with strategy:depend when latest child pipeline failed does not unlock job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb	129	0	0%
Name: Verify Unlocking job artifacts across parent-child pipelines with strategy:depend when latest parent pipeline failed does not unlock job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb	124	0	0%

Blocking specs with failures (2024-07-22 - 2024-08-05)

Note:

• Only failures from the nightly, e2e-package-and-test and e2e-test-on-gdk pipelines are considered

• Only specs that have a failure rate of equal or greater than 1 percent are considered

• Quarantine MRs will be created for all specs listed below

Total amount: 4

STAGE	COUNT
data_stores	1
package	3

Data_stores (1)

Executions table grouptenant scale (1)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Data Stores Subgroup transfer transfers a subgroup to another group File: transfer_group_spec.rb Exceptions: - expected to find text "Group 'subgroup-for-transfer_c4bee5b925d8b1a0' was successfully transferred." in "Skip to main content Primary navigation Homepage Create new... Administrator user’s menu Search or go to… Group S subgroup-for-transfer_c4... - expected to find text "Group 'subgroup-for-transfer_311e4e0965d3a13d' was successfully transferred." in "Skip to main content Primary navigation Homepage Create new... Administrator user’s menu 4 3 Search or go to… Group S subgroup-for-transfe... - expected to find text "Group 'subgroup-for-transfer_ba6ca8cd4febed64' was successfully transferred." in "Skip to main content Primary navigation Homepage Create new... Administrator user’s menu Search or go to… Group S subgroup-for-transfer_ba...	207	3	1.45%

Package (3)

Executions table grouppackage registry (3)

NAME	RUNS	FAILURES	FAILURE RATE
Name: Package npm Registry group level endpoint using ci job token push and pull a npm package via CI File: npm_group_level_spec.rb Exceptions: - Wait for any pipeline failed after 120 seconds - Resource at https://staging.gitlab.com/api/v4/projects/13172624?private_token=[****] could not be deleted (502): '<!DOCTYPE html><html><head> <meta content="width=device-width, initial-scale=1, maximum-scale=1" name="viewport"> <title>500 Error ...	404	8	1.98%
Name: Package npm Registry group level endpoint using personal access token push and pull a npm package via CI File: npm_group_level_spec.rb Exceptions: - Wait for any pipeline failed after 120 seconds	404	7	1.73%
Name: Package npm Registry instance level endpoint using personal access token push and pull a npm package via CI File: npm_instance_level_spec.rb Exceptions: - RSpec::Core::MultipleExceptionError - Wait for any pipeline failed after 120 seconds	297	3	1.01%