Reliable e2e test report
- Candidates for promotion to reliable/blocking (2024-04-08 - 2024-04-22)
- Reliable specs with failures (2024-04-08 - 2024-04-22)
Candidates for promotion to reliable/blocking (2024-04-08 - 2024-04-22)
Note: MRs will be auto-created for promoting the top 10 specs sorted by most number of successful runs
Total amount: 87
STAGE | COUNT |
---|---|
analytics | 1 |
data_stores | 13 |
create | 28 |
fulfillment | 2 |
monitor | 3 |
govern | 13 |
manage | 4 |
plan | 11 |
systems | 3 |
verify | 9 |
Analytics (1)
Executions table groupanalytics instrumentation (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Analytics Service ping disabled when disabled from gitlab.yml config has service ping toggle is disabled File: service_ping_disabled_spec.rb |
170 | 0 | 0% |
Data_stores (13)
Executions table grouptenant scale (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Data Stores Group with members can be shared with another group with correct access level File: share_group_with_group_spec.rb |
517 | 0 | 0% |
Name: Configure Mattermost support user creates a group with a mattermost team File: create_group_with_mattermost_team_spec.rb |
169 | 0 | 0% |
Executions table groupglobal search (11)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Data Stores When using elasticsearch API to search for a known blob searches public project and finds a blob as an non-member user File: elasticsearch_api_spec.rb |
290 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a known blob When searching a private repository does not find a blob as an non-member user File: elasticsearch_api_spec.rb |
225 | 0 | 0% |
Name: Data Stores When using Advanced Search API to search for a public commit finds commit that matches commit message File: commit_index_spec.rb |
225 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public merge request finds merge request that matches description File: merge_request_index_spec.rb |
225 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public issue finds issue that matches description File: issue_index_spec.rb |
225 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public note finds note that matches note body File: note_index_spec.rb |
225 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a known blob When searching a private repository finds a blob as an authorized user File: elasticsearch_api_spec.rb |
225 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public blob finds blob that matches file content File: blob_index_spec.rb |
225 | 0 | 0% |
Name: Data Stores When using advanced search API to search for a user finds the user that matches username File: user_index_spec.rb |
217 | 0 | 0% |
Name: Data Stores Search using Elasticsearch tests reindexing after webIDE File: elasticsearch_reindexing_spec.rb |
217 | 0 | 0% |
Name: Data Stores Search using Elasticsearch tests reindexing after push File: elasticsearch_reindexing_spec.rb |
217 | 0 | 0% |
Create (28)
Executions table groupide (8)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Upload a file in Web IDE when the file is an image behaves like upload a file verifies it successfully uploads and commits to a MR File: upload_new_file_in_web_ide_spec.rb |
690 | 0 | 0% |
Name: Create Closing Web IDE shows an alert when there are unsaved changes File: closing_web_ide_with_unsaved_changes_spec.rb |
687 | 0 | 0% |
Name: Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed File: add_new_directory_in_web_ide_spec.rb |
686 | 0 | 0% |
Name: Create Add a directory in Web IDE when a directory with the same name already exists throws an error File: add_new_directory_in_web_ide_spec.rb |
686 | 0 | 0% |
Name: Create Code Suggestions in Web IDE on Self-managed with no license behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb |
170 | 0 | 0% |
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with no Duo Pro add-on behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb |
169 | 0 | 0% |
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with a Duo Pro add-on when seat is not assigned behaves like unauthorized returns no suggestion File: code_suggestions_in_web_ide_spec.rb |
165 | 0 | 0% |
Name: Create Code Suggestions in Web IDE on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like a code completion suggestion returns a code completion suggestion which can be accepted File: code_suggestions_in_web_ide_spec.rb |
164 | 0 | 0% |
Executions table groupsource code (13)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Push mirror a repository over HTTP configures and syncs a (push) mirrored repository File: push_mirroring_over_http_spec.rb |
854 | 0 | 0% |
Name: Create Adding comments on snippets behaves like comments on snippets adds, edits, and deletes a comment on a project_snippet File: add_comment_to_snippet_spec.rb |
693 | 0 | 0% |
Name: Create Adding comments on snippets behaves like comments on snippets adds, edits, and deletes a comment on a personal_snippet File: add_comment_to_snippet_spec.rb |
693 | 0 | 0% |
Name: Create File Locking locks a file and unlocks in list File: file_locking_spec.rb |
690 | 0 | 0% |
Name: Create File Locking locks a directory and tries to push as a second user File: file_locking_spec.rb |
690 | 0 | 0% |
Name: Create File with unusual name when file name starts with a dash and contains hash, semicolon, colon, and question mark renders repository file tree correctly File: file_with_unusual_name_spec.rb |
690 | 0 | 0% |
Name: Create Multiple file snippet behaves like adding file to snippet adds second file to an existing project_snippet to make it multi-file File: add_file_to_snippet_spec.rb |
690 | 0 | 0% |
Name: Create Multiple file snippet behaves like adding file to snippet adds second file to an existing personal_snippet to make it multi-file File: add_file_to_snippet_spec.rb |
690 | 0 | 0% |
Name: Create Push over HTTP using Git protocol version 2 user pushes to the repository File: protocol_v2_push_http_spec.rb |
688 | 0 | 0% |
Name: Create Repository License Detection on a project with a commonly used LICENSE behaves like project license detection displays the name of the license on the repository File: license_detection_spec.rb |
687 | 0 | 0% |
Name: Create Repository License Detection on a project with an unrecognized LICENSE behaves like project license detection displays the name of the license on the repository File: license_detection_spec.rb |
687 | 0 | 0% |
Name: Create Push over SSH using Git protocol version 2 user pushes to the repository File: protocol_v2_push_ssh_spec.rb |
683 | 0 | 0% |
Name: Create Prereceive hook when creating a tag for a ref when it triggers a prereceive hook configured with a custom error returns a custom server hook error File: tag_revision_trigger_prereceive_hook_spec.rb |
676 | 0 | 0% |
Executions table groupcode creation (4)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Code Suggestions when code completion is requested on Self-managed with no license behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb |
170 | 0 | 0% |
Name: Create Code Suggestions when code completion is requested on Self-managed with a valid license with no Duo Pro add-on behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb |
169 | 0 | 0% |
Name: Create Code Suggestions when code completion is requested on Self-managed with a valid license with a Duo Pro add-on when seat is not assigned behaves like unauthorized returns no suggestion File: code_suggestions_spec.rb |
165 | 0 | 0% |
Name: Create Code Suggestions when code completion is requested on Self-managed with a valid license with a Duo Pro add-on when seat is assigned behaves like completions API with PAT auth returns a suggestion File: code_suggestions_spec.rb |
164 | 0 | 0% |
Executions table groupcode review (3)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Merge request creation from fork can merge source branch from fork into upstream repository File: merge_merge_request_from_fork_spec.rb |
689 | 0 | 0% |
Name: Create Merge request when merging once behaves like set to auto-merge merges after pipeline succeeds File: merge_request_set_to_auto_merge_spec.rb |
174 | 0 | 0% |
Name: Create Merge Requests with a malformed URL returns 400 File: view_merge_requests_spec.rb |
166 | 0 | 0% |
Fulfillment (2)
Executions table groupprovision (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Fulfillment Active license details shows up in subscription page File: license_spec.rb |
513 | 0 | 0% |
Executions table groupsubscription management (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Fulfillment Purchase User Registration when adding and removing a group member consumes a seat on the license File: user_registration_billing_spec.rb |
510 | 0 | 0% |
Monitor (3)
Executions table groupobservability (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: GitLab Metrics Web metrics via dedicated server returns 200 OK and serves metrics File: metrics_spec.rb |
168 | 0 | 0% |
Executions table grouprespond (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Monitor Recovery alert when using HTTP endpoint integration behaves like triggers recovery alert only resolves the correct alert File: recovery_alert_resolves_correct_alert_spec.rb |
622 | 0 | 0% |
Name: Monitor Recovery alert when using Prometheus integration behaves like triggers recovery alert only resolves the correct alert File: recovery_alert_resolves_correct_alert_spec.rb |
166 | 0 | 0% |
Govern (13)
Executions table groupauthentication (12)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern User with minimal access to group is not allowed to push code via the CLI File: minimal_access_user_spec.rb |
521 | 0 | 0% |
Name: Govern LDAP login user logs into GitLab using LDAP credentials File: log_into_gitlab_via_ldap_spec.rb |
339 | 0 | 0% |
Name: Govern LDAP Group sync with user filter method behaves like Group sync can create project access tokens File: group_ldap_sync_spec.rb |
339 | 0 | 0% |
Name: Govern LDAP Group sync with group cn method behaves like Group sync can create group access tokens File: group_ldap_sync_spec.rb |
339 | 0 | 0% |
Name: Govern LDAP Group sync with group cn method behaves like Group sync can create project access tokens File: group_ldap_sync_spec.rb |
339 | 0 | 0% |
Name: Govern LDAP Group sync with user filter method behaves like Group sync can create group access tokens File: group_ldap_sync_spec.rb |
339 | 0 | 0% |
Name: Govern Group SAML SSO - Non enforced SSO user logs in to group with SAML SSO File: group_saml_non_enforced_sso_spec.rb |
170 | 0 | 0% |
Name: Govern Group SAML SSO - Non enforced SSO lets group admin test settings File: group_saml_non_enforced_sso_spec.rb |
170 | 0 | 0% |
Name: Govern Group SAML SSO - Non enforced SSO when SAML SSO is configured with a default membership role adds the new member with access level as set in SAML SSO configuration File: group_saml_non_enforced_sso_spec.rb |
170 | 0 | 0% |
Name: Govern OAuth connects and logs in with GitHub OAuth File: oauth_login_with_github_spec.rb |
170 | 0 | 0% |
Name: Govern Mattermost login user logs into Mattermost using GitLab OAuth File: log_into_mattermost_via_gitlab_spec.rb |
169 | 0 | 0% |
Name: Govern Instance wide SAML SSO user logs in to gitlab with SAML SSO File: login_via_instance_wide_saml_sso_spec.rb |
168 | 0 | 0% |
Executions table groupthreat insights (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern Vulnerability Report Security Training does not show security training section in vulnerability details when the setting is turned OFF File: vulnerability_security_training_spec.rb |
525 | 0 | 0% |
Manage (4)
Executions table groupimport and integrate (4)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Manage Jenkins integration integrates and displays build status for MR pipeline in GitLab File: jenkins_build_status_spec.rb |
513 | 0 | 0% |
Name: Manage Jira issues integration searching issues returns results File: jira_issues_list_spec.rb |
169 | 0 | 0% |
Name: Manage Jira issues integration shows open issues File: jira_issues_list_spec.rb |
169 | 0 | 0% |
Name: Manage Jira issues integration views an issue File: jira_issues_list_spec.rb |
169 | 0 | 0% |
Plan (11)
Executions table groupknowledge (6)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Testing group wiki can delete a group wiki page File: delete_group_wiki_page_spec.rb |
523 | 0 | 0% |
Name: Plan Testing wiki content creation inside a project by adding a home page to the wiki File: project_based_content_creation_spec.rb |
522 | 0 | 0% |
Name: Plan Testing wiki content creation inside a project by adding a home page to the wiki using git push File: project_based_content_creation_spec.rb |
522 | 0 | 0% |
Name: Plan Testing wiki content creation inside a project by adding a second page to the wiki File: project_based_content_creation_spec.rb |
522 | 0 | 0% |
Name: Plan Testing wiki content creation inside a project by adding a second page to the wiki using git push File: project_based_content_creation_spec.rb |
522 | 0 | 0% |
Name: Plan Testing wiki content creation inside a project by adding a wiki page with spaces in the path using git push File: project_based_content_creation_spec.rb |
522 | 0 | 0% |
Executions table groupproject management (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Issue closes via pushing a commit File: closes_issue_via_pushing_a_commit_spec.rb |
521 | 0 | 0% |
Name: Plan Custom email customizes email with additional text File: custom_email_spec.rb |
520 | 0 | 0% |
Executions table groupproduct planning (3)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Design Management user archives a design File: archive_design_content_spec.rb |
524 | 0 | 0% |
Name: Plan Design Management user adds a design and modifies it File: modify_design_content_spec.rb |
521 | 0 | 0% |
Name: Plan Epics roadmap presents epic on roadmap File: roadmap_spec.rb |
166 | 0 | 0% |
Systems (3)
Executions table groupgitaly (3)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Systems Gitaly Cluster replication queue allows replication of different repository after interruption File: praefect_replication_queue_spec.rb |
170 | 0 | 0% |
Name: Systems Gitaly distributed reads reads from each node File: distributed_reads_spec.rb |
170 | 0 | 0% |
Name: Systems Gitaly distributed reads when a node is unhealthy does not read from the unhealthy node File: distributed_reads_spec.rb |
170 | 0 | 0% |
Verify (9)
Executions table grouprunner (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Runner fleet management shows group runner online count File: group_runner_status_counts_spec.rb |
526 | 0 | 0% |
Executions table grouppipeline security (6)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify UI defined variable is not inheritable when forward:pipeline_variables is false File: ui_variable_non_inheritable_when_forward_pipeline_variables_false_spec.rb |
523 | 0 | 0% |
Name: Verify UI defined variable is not inheritable by default File: ui_variable_non_inheritable_when_forward_pipeline_variables_false_spec.rb |
523 | 0 | 0% |
Name: Verify Job artifacts when exposed show up in MR widget File: expose_job_artifacts_in_mr_spec.rb |
520 | 0 | 0% |
Name: Verify UI defined variable is inheritable when forward:pipeline_variables is true File: ui_variable_inheritable_when_forward_pipeline_variables_true_spec.rb |
518 | 0 | 0% |
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest parent pipeline failed does not unlock job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
200 | 0 | 0% |
Name: Verify Unlocking job artifacts across parent-child pipelines without strategy:depend when latest pipeline family is successful unlocks job artifacts from previous successful pipeline family File: unlocking_job_artifacts_across_parent_child_pipelines_spec.rb |
199 | 0 | 0% |
Executions table grouppipeline execution (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Trigger child pipeline with 'when:manual' can trigger bridge job File: trigger_child_pipeline_with_manual_spec.rb |
520 | 0 | 0% |
Name: Verify when pipeline is blocked can still merge MR successfully File: merge_mr_when_pipline_is_blocked_spec.rb |
166 | 0 | 0% |
Reliable specs with failures (2024-04-08 - 2024-04-22)
Note:
-
Only failures from the nightly, e2e-package-and-test and e2e-test-on-gdk pipelines are considered
-
Only specs that have a failure rate of equal or greater than 1 percent are considered
-
Quarantine MRs will be created for all specs listed below
Total amount: 14
STAGE | COUNT |
---|---|
govern | 1 |
manage | 2 |
package | 11 |
Govern (1)
Executions table groupcompliance (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern Instance for start and stop user impersonation behaves like audit event logs audit events for UI operations File: instance_audit_logs_spec.rb Exceptions: - invalid argument (Session info: chrome=120.0.6099.216) - undefined method 'submit_search_term' for QA::Page::Admin::Overview::Users::Index:QA::Page::Admin::Overview::Users::Index submit_search_term(username) ^^^^^^^^^^^^^^^^^^ - <code data-sourcepos="300:713-300:833">undefined method 'submit_search_term' for QA::Page::Admin::Overview::Users::Index:QA::Page::Admin::Overview::Users::Index</code> |
664 | 8 | 1.2% |