Reliable e2e test report
- Candidates for promotion to blocking (2024-07-01 - 2024-07-15)
- Blocking specs with failures (2024-07-01 - 2024-07-15)
Candidates for promotion to blocking (2024-07-01 - 2024-07-15)
Note: MRs will be auto-created for promoting the top 10 specs sorted by most number of successful runs
Total amount: 89
STAGE | COUNT |
---|---|
analytics | 1 |
data_stores | 6 |
create | 15 |
fulfillment | 1 |
monitor | 9 |
govern | 16 |
manage | 18 |
package | 1 |
plan | 5 |
release | 1 |
secure | 1 |
systems | 7 |
verify | 8 |
Analytics (1)
Executions table groupanalytics instrumentation (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Analytics Service ping disabled when disabled from gitlab.yml config has service ping toggle is disabled File: service_ping_disabled_spec.rb |
166 | 0 | 0% |
Data_stores (6)
Executions table grouptenant scale (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Configure Mattermost support user creates a group with a mattermost team File: create_group_with_mattermost_team_spec.rb |
166 | 0 | 0% |
Executions table groupglobal search (5)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Data Stores When using elasticsearch API to search for a public issue finds issue that matches description File: issue_index_spec.rb |
225 | 0 | 0% |
Name: Data Stores Search using Elasticsearch tests reindexing after push File: elasticsearch_reindexing_spec.rb |
214 | 0 | 0% |
Name: Data Stores Search using Elasticsearch tests reindexing after webIDE File: elasticsearch_reindexing_spec.rb |
214 | 0 | 0% |
Name: Data Stores When using advanced search API to search for a user finds the user that matches username File: user_index_spec.rb |
214 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a known blob searches public project and finds a blob as an non-member user File: nightly_elasticsearch_test_spec.rb |
60 | 0 | 0% |
Create (15)
Executions table groupide (5)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Code Suggestions in Web IDE on Self-managed with no license behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb |
166 | 0 | 0% |
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with no Duo Pro add-on behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb |
165 | 0 | 0% |
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with a Duo Pro add-on when seat is not assigned behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb |
154 | 0 | 0% |
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like a code completion suggestion returns a code completion suggestion which can be accepted File: code_suggestions_in_web_ide_spec.rb |
150 | 0 | 0% |
Name: Create Duo Chat in Web IDE on Self-managed behaves like Duo Chat gets a response back from Duo Chat File: duo_chat_in_web_ide_spec.rb |
150 | 0 | 0% |
Executions table groupcode creation (6)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Code Suggestions when code generation is requested on Self-managed with no license behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb |
166 | 0 | 0% |
Name: Create Code Suggestions when code generation is requested on Self-managed with a valid license with no Duo Pro add-on behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb |
165 | 0 | 0% |
Name: Create Code Suggestions when code generation is requested on Self-managed with a valid license with a Duo Pro add-on when seat is not assigned behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb |
154 | 0 | 0% |
Name: Create Code Suggestions when code completion on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like code suggestions API returns a suggestion File: code_suggestions_spec.rb |
150 | 0 | 0% |
Name: Create Code Suggestions when code generation is requested on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like code suggestions API returns a suggestion File: code_suggestions_spec.rb |
150 | 0 | 0% |
Name: Create Code Suggestions when code generation is requested when streaming on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like code suggestions API using streaming streams a suggestion File: code_suggestions_spec.rb |
150 | 0 | 0% |
Executions table groupcode review (4)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Merge Requests with a malformed URL returns 400 File: view_merge_requests_spec.rb |
165 | 0 | 0% |
Name: Create Merge request when merging once behaves like set to auto-merge merges after pipeline succeeds File: merge_request_set_to_auto_merge_spec.rb |
157 | 0 | 0% |
Name: Create Merge request when merging several times behaves like set to auto-merge merges after pipeline succeeds File: merge_request_set_to_auto_merge_spec.rb |
141 | 0 | 0% |
Name: Create Merge request push options merges when pipeline succeeds File: push_options_mwps_spec.rb |
72 | 0 | 0% |
Fulfillment (1)
Executions table groupprovision (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Fulfillment Cloud activation code activates instance with correct subscription details File: cloud_activation_spec.rb |
165 | 0 | 0% |
Monitor (9)
Executions table groupobservability (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: GitLab Metrics Sidekiq metrics via dedicated server returns 200 OK and serves metrics File: metrics_spec.rb |
166 | 0 | 0% |
Name: GitLab Metrics Web metrics via dedicated server returns 200 OK and serves metrics File: metrics_spec.rb |
166 | 0 | 0% |
Executions table grouprespond (7)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Monitor Alert when user is a maintainer when using HTTP endpoint integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
166 | 0 | 0% |
Name: Monitor Alert when user is a maintainer when using Prometheus integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
166 | 0 | 0% |
Name: Monitor Alert when user is an owner when using HTTP endpoint integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
166 | 0 | 0% |
Name: Monitor Alert when user is an owner when using Prometheus integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
166 | 0 | 0% |
Name: Monitor Recovery alert when using Prometheus integration behaves like triggers recovery alert only resolves the correct alert File: recovery_alert_resolves_correct_alert_spec.rb |
161 | 0 | 0% |
Name: Monitor Recovery alert when using HTTP endpoint integration behaves like triggers recovery alert only closes the correct incident File: recovery_alert_closes_correct_incident_spec.rb |
159 | 0 | 0% |
Name: Monitor Recovery alert when using Prometheus integration behaves like triggers recovery alert only closes the correct incident File: recovery_alert_closes_correct_incident_spec.rb |
150 | 0 | 0% |
Govern (16)
Executions table groupauthentication (14)
Executions table groupcompliance (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern Merge request approval with SAML SSO can be approved when requiring a password File: saml_sso_merge_request_approve_spec.rb |
164 | 0 | 0% |
Executions table groupthreat insights (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern vulnerability report with jira integration can successfully create a JIRA issue from vulnerability details page File: vulnerabilities_jira_integration_spec.rb |
165 | 0 | 0% |
Manage (18)
Executions table groupimport and integrate (18)
Package (1)
Executions table groupcontainer registry (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Package Dependency Proxy Group SSO pulls an image using the dependency proxy on a group enforced SSO File: dependency_proxy_sso_spec.rb |
164 | 0 | 0% |
Plan (5)
Executions table groupproject management (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Email Notification is received by a user for project invitation File: trigger_email_notification_spec.rb |
166 | 0 | 0% |
Executions table groupknowledge (3)
Executions table groupproduct planning (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Epics roadmap presents epic on roadmap File: roadmap_spec.rb |
162 | 0 | 0% |
Release (1)
Executions table groupenvironments (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Release Operations Dashboard has many pipelines with appropriate statuses File: pipeline_status_on_operation_dashboard_spec.rb |
159 | 0 | 0% |
Secure (1)
Executions table groupstatic analysis (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Secure Enable Scanning from UI enable sast from configuration runs sast job when enabled from configuration File: enable_scanning_from_configuration_spec.rb Exceptions: - Failed to GET http://gitlab-ee-abf5baa1.test/api/v4/groups/gitlab-qa-sandbox-group-2%2Fqa-test-2024-07-08-10-15-56-9bae55e73290c751?with_projects=false&private_token=[****] - (500): '<!DOCTYPE html><html><head> <meta content="width=device-width, ... - Failed to GET http://gitlab-ee-64eea4d2.test/api/v4/groups/gitlab-qa-sandbox-group-2%2Fqa-test-2024-07-08-18-56-07-506c6581f372a69a?with_projects=false&private_token=[****] - (500): '<!DOCTYPE html><html><head> <meta content="width=device-width, ... - <code data-sourcepos="253:977-253:1064">Page did not fully load. This could be due to an unending async request or loading icon.</code> |
219 | 0 | 0% |
Systems (7)
Executions table groupgitaly (7)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Systems Gitaly Cluster replication queue allows replication of different repository after interruption File: praefect_replication_queue_spec.rb |
164 | 0 | 0% |
Name: Systems Gitaly backend node recovery recovers from dataloss File: backend_node_recovery_spec.rb |
164 | 0 | 0% |
Name: Systems Gitaly distributed reads reads from each node File: distributed_reads_spec.rb |
164 | 0 | 0% |
Name: Systems Gitaly distributed reads when a node is unhealthy does not read from the unhealthy node File: distributed_reads_spec.rb |
164 | 0 | 0% |
Name: Systems Gitaly using mTLS pushes to gitaly File: gitaly_mtls_spec.rb |
164 | 0 | 0% |
Name: Systems with Gitaly automatic failover and recovery automatically fails over File: automatic_failover_and_recovery_spec.rb |
164 | 0 | 0% |
Name: Systems with Gitaly automatic failover and recovery when recovering from dataloss after failover automatically reconciles File: automatic_failover_and_recovery_spec.rb |
164 | 0 | 0% |
Verify (8)
Executions table grouppipeline authoring (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify CI catalog when sorting with released at behaves like descending order by default displays from last to first File: ci_catalog_sorting_spec.rb |
159 | 0 | 0% |
Executions table grouppipeline execution (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify when pipeline is blocked can still merge MR successfully File: merge_mr_when_pipline_is_blocked_spec.rb |
162 | 0 | 0% |
Executions table grouppipeline security (6)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Unlocking job artifacts across parent-child pipelines with strategy:depend when latest pipeline family is successful unlocks job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
158 | 0 | 0% |
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest child pipeline failed unlocks job artifacts from previous successful pipeline family because the latest parent is successful File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
158 | 0 | 0% |
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest parent pipeline failed does not unlock job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
158 | 0 | 0% |
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest pipeline family is successful unlocks job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
158 | 0 | 0% |
Name: Verify Unlocking job artifacts across parent-child pipelines with strategy:depend when latest parent pipeline failed does not unlock job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
123 | 0 | 0% |
Name: Verify Unlocking job artifacts across parent-child pipelines with strategy:depend when latest child pipeline failed does not unlock job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
122 | 0 | 0% |
Blocking specs with failures (2024-07-01 - 2024-07-15)
Note:
-
Only failures from the nightly, e2e-package-and-test and e2e-test-on-gdk pipelines are considered
-
Only specs that have a failure rate of equal or greater than 1 percent are considered
-
Quarantine MRs will be created for all specs listed below
Total amount: 26
STAGE | COUNT |
---|---|
create | 7 |
data_stores | 2 |
package | 11 |
plan | 1 |
verify | 5 |
Create (7)
Executions table groupsource code (6)
Executions table groupcode review (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Merge request creation from fork can merge source branch from fork into upstream repository File: merge_merge_request_from_fork_spec.rb Exceptions: - RSpec::Core::MultipleExceptionError
|
723 | 10 | 1.38% |
Data_stores (2)
Executions table grouptenant scale (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Data Stores Invite group with a personal namespace project behaves like invites group to project grants group and members correct access level File: invite_group_to_project_spec.rb Exceptions: - RSpec::Core::MultipleExceptionError - Unable to find css "[data-testid=\"flash-container\"]" - Unable to find css "[data-testid=\"project-content\"]"
|
545 | 7 | 1.28% |
Name: Data Stores Invite group with a group project behaves like invites group to project grants group and members correct access level File: invite_group_to_project_spec.rb Exceptions: - Unable to find css "[data-testid=\"project-content\"]" - RSpec::Core::MultipleExceptionError - Failed to GET http://gitlab-ee-d0004332.test/api/v4/groups/gitlab-qa-sandbox-group-3?with_projects=false&private_token=[****] - (500): '<!DOCTYPE html><html><head> <meta content="width=device-width, initial-scale=1" name="viewport"> <title>Somet... - Unable to find css "[data-testid=\"flash-container\"]"
|
545 | 6 | 1.1% |
Package (11)
Executions table grouppackage registry (11)
Plan (1)
Executions table groupknowledge (1)
Verify (5)
Executions table grouppipeline authoring (2)
Executions table grouppipeline execution (2)
Executions table grouppipeline security (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Unlocking job artifacts across pipelines when latest pipeline is blocked keeps job artifacts from the latest blocked pipeline and from latest successful pipeline File: unlocking_job_artifacts_across_pipelines_spec.rb Exceptions: - <code data-sourcepos="414:431-414:478">Wait for latest pipeline failed after 60 seconds</code> - RSpec::Core::MultipleExceptionError - Wait failed after 60 seconds - Failed to GET http://gitlab-ee-642a3c19.test/api/v4/projects/gitlab-qa-sandbox-group-3%2Fqa-test-2024-07-09-16-59-40-3845615aac0b7a48%2Funlock-job-artifacts-project-d62267260385ed47/runners?tag_list=qa-runner-c8g9qel2&private_token=[****] - (500):...
|
393 | 5 | 1.27% |