Reliable e2e test report
Candidates for promotion to reliable/blocking (2024-03-18 - 2024-04-01)
Note: MRs will be auto-created for promoting the top 10 specs sorted by most number of successful runs
Total amount: 130
STAGE | COUNT |
---|---|
analytics | 1 |
data_stores | 15 |
create | 16 |
fulfillment | 1 |
monitor | 6 |
govern | 24 |
manage | 27 |
package | 14 |
plan | 8 |
systems | 9 |
verify | 9 |
Analytics (1)
Executions table groupanalytics instrumentation (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Analytics Service ping disabled when disabled from gitlab.yml config has service ping toggle is disabled File: service_ping_disabled_spec.rb |
175 | 0 | 0% |
Data_stores (15)
Executions table grouptenant scale (4)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Data Stores prevent forking outside group when disabled allows forking outside of group File: prevent_forking_outside_group_spec.rb |
550 | 0 | 0% |
Name: Data Stores Group with members can be shared with another group with correct access level File: share_group_with_group_spec.rb |
550 | 0 | 0% |
Name: Data Stores prevent forking outside group when enabled does not allow forking outside of group File: prevent_forking_outside_group_spec.rb |
550 | 0 | 0% |
Name: Configure Mattermost support user creates a group with a mattermost team File: create_group_with_mattermost_team_spec.rb |
175 | 0 | 0% |
Executions table groupglobal search (11)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Data Stores When using elasticsearch API to search for a known blob searches public project and finds a blob as an non-member user File: elasticsearch_api_spec.rb |
301 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a known blob When searching a private repository does not find a blob as an non-member user File: elasticsearch_api_spec.rb |
236 | 0 | 0% |
Name: Data Stores When using Advanced Search API to search for a public commit finds commit that matches commit message File: commit_index_spec.rb |
236 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public merge request finds merge request that matches description File: merge_request_index_spec.rb |
236 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public issue finds issue that matches description File: issue_index_spec.rb |
236 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public note finds note that matches note body File: note_index_spec.rb |
236 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a known blob When searching a private repository finds a blob as an authorized user File: elasticsearch_api_spec.rb |
236 | 0 | 0% |
Name: Data Stores When using elasticsearch API to search for a public blob finds blob that matches file content File: blob_index_spec.rb |
236 | 0 | 0% |
Name: Data Stores When using advanced search API to search for a user finds the user that matches username File: user_index_spec.rb |
227 | 0 | 0% |
Name: Data Stores Search using Elasticsearch tests reindexing after webIDE File: elasticsearch_reindexing_spec.rb |
227 | 0 | 0% |
Name: Data Stores Search using Elasticsearch tests reindexing after push File: elasticsearch_reindexing_spec.rb |
227 | 0 | 0% |
Create (16)
Executions table groupsource code (11)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Pull mirror a repository over SSH with a private key configures and syncs a (pull) mirrored repository File: pull_mirroring_over_ssh_with_key_spec.rb |
735 | 0 | 0% |
Name: Create Branch with unusual name when branch name contains slash, hash, double dash, and capital letter renders repository file tree correctly File: branch_with_unusual_name_spec.rb |
735 | 0 | 0% |
Name: Create Push Rules using non signed commits restricts removal of tag File: push_rules_spec.rb |
735 | 0 | 0% |
Name: Create Push Rules with commits restricted to verified emails rejects unverified emails File: push_rules_spec.rb |
735 | 0 | 0% |
Name: Create Project snippet creation user creates a project snippet File: create_project_snippet_spec.rb |
734 | 0 | 0% |
Name: Create Source editor toolbar preview can preview markdown side-by-side while editing File: source_editor_toolbar_spec.rb |
734 | 0 | 0% |
Name: Create API basics user creates a project with a file and deletes them afterwards File: files_spec.rb |
733 | 0 | 0% |
Name: Create Push over SSH using Git protocol version 2 user pushes to the repository File: protocol_v2_push_ssh_spec.rb |
733 | 0 | 0% |
Name: Create Setup an MR with codeowners file creates a merge request with codeowners file and squashing commits enabled File: code_owners_with_protected_branch_and_squashed_commits_spec.rb |
733 | 0 | 0% |
Name: Create Gitaly repository storage creates a 2nd fork after moving the parent project File: move_project_create_fork_spec.rb |
175 | 0 | 0% |
Name: Create Snippet repository storage moves snippet repository from one Gitaly storage to another File: snippet_repository_storage_move_spec.rb |
175 | 0 | 0% |
Executions table groupide (3)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Upload a file in Web IDE when a file with the same name already exists throws an error File: upload_new_file_in_web_ide_spec.rb |
734 | 0 | 0% |
Name: Create Upload a file in Web IDE when the file is a text file behaves like upload a file verifies it successfully uploads and commits to a MR File: upload_new_file_in_web_ide_spec.rb |
734 | 0 | 0% |
Name: Create Upload a file in Web IDE when the file is an image behaves like upload a file verifies it successfully uploads and commits to a MR File: upload_new_file_in_web_ide_spec.rb |
734 | 0 | 0% |
Executions table ~"group::editor" (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Group Wiki repository storage moves group Wiki repository from one Gitaly storage to another File: group_wiki_repository_storage_move_spec.rb |
175 | 0 | 0% |
Executions table groupcode review (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Create Merge request rebasing user rebases source branch of merge request File: rebase_merge_request_spec.rb |
733 | 0 | 0% |
Fulfillment (1)
Executions table groupprovision (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Fulfillment Cloud activation code activates instance with correct subscription details File: cloud_activation_spec.rb |
175 | 0 | 0% |
Monitor (6)
Executions table groupobservability (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: GitLab Metrics Sidekiq metrics via dedicated server returns 200 OK and serves metrics File: metrics_spec.rb |
175 | 0 | 0% |
Name: GitLab Metrics Web metrics via dedicated server returns 200 OK and serves metrics File: metrics_spec.rb |
175 | 0 | 0% |
Executions table grouprespond (4)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Monitor Alert when user is a maintainer when using HTTP endpoint integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
175 | 0 | 0% |
Name: Monitor Alert when user is a maintainer when using Prometheus integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
175 | 0 | 0% |
Name: Monitor Alert when user is an owner when using HTTP endpoint integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
175 | 0 | 0% |
Name: Monitor Alert when user is an owner when using Prometheus integration behaves like notification on new alert sends email to user File: email_notification_for_alert_spec.rb |
175 | 0 | 0% |
Govern (24)
Executions table groupauthentication (21)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern User with minimal access to group is not allowed to push code via the CLI File: minimal_access_user_spec.rb |
558 | 0 | 0% |
Name: Govern User when parent group membership is terminated is not allowed to create a file via the API File: user_access_termination_spec.rb |
551 | 0 | 0% |
Name: Govern User when parent group membership is terminated is not allowed to push code via the CLI File: user_access_termination_spec.rb |
551 | 0 | 0% |
Name: Govern User when parent group membership is terminated is not allowed to commit via the API File: user_access_termination_spec.rb |
551 | 0 | 0% |
Name: Govern LDAP Group sync with group cn method behaves like Group sync can create project access tokens File: group_ldap_sync_spec.rb |
349 | 0 | 0% |
Name: Govern LDAP Group sync with group cn method behaves like Group sync can create group access tokens File: group_ldap_sync_spec.rb |
349 | 0 | 0% |
Name: Govern LDAP login user logs into GitLab using LDAP credentials File: log_into_gitlab_via_ldap_spec.rb |
349 | 0 | 0% |
Name: Govern LDAP Group sync with user filter method behaves like Group sync can create project access tokens File: group_ldap_sync_spec.rb |
349 | 0 | 0% |
Name: Govern LDAP Group sync with user filter method behaves like Group sync has LDAP users synced File: group_ldap_sync_spec.rb |
349 | 0 | 0% |
Name: Govern LDAP Group sync with group cn method behaves like Group sync has LDAP users synced File: group_ldap_sync_spec.rb |
349 | 0 | 0% |
Name: Govern LDAP Group sync with user filter method behaves like Group sync can create group access tokens File: group_ldap_sync_spec.rb |
349 | 0 | 0% |
Name: Govern Instance wide SAML SSO user logs in to gitlab with SAML SSO File: login_via_instance_wide_saml_sso_spec.rb |
175 | 0 | 0% |
Name: Govern Group SAML SSO - Non enforced SSO when SAML SSO is configured with a default membership role adds the new member with access level as set in SAML SSO configuration File: group_saml_non_enforced_sso_spec.rb |
175 | 0 | 0% |
Name: Govern Group SAML SSO - Non enforced SSO lets group admin test settings File: group_saml_non_enforced_sso_spec.rb |
175 | 0 | 0% |
Name: Govern Group SAML SSO - Enforced SSO with Snowplow tracking disabled behaves like group membership actions creates a new account automatically and allows to leave group and join again File: group_saml_enforced_sso_new_account_spec.rb |
175 | 0 | 0% |
Name: Govern Group SAML SSO - Enforced SSO user clones and pushes to project within a group using Git HTTP File: group_saml_enforced_sso_git_access_spec.rb |
175 | 0 | 0% |
Name: Govern Group SAML SSO - Enforced SSO with Snowplow tracking enabled behaves like group membership actions creates a new account automatically and allows to leave group and join again File: group_saml_enforced_sso_new_account_spec.rb |
175 | 0 | 0% |
Name: Govern Group SAML SSO - Non enforced SSO user logs in to group with SAML SSO File: group_saml_non_enforced_sso_spec.rb |
175 | 0 | 0% |
Name: Govern Mattermost login user logs into Mattermost using GitLab OAuth File: log_into_mattermost_via_gitlab_spec.rb |
175 | 0 | 0% |
Name: Govern OAuth connects and logs in with GitHub OAuth File: oauth_login_with_github_spec.rb |
174 | 0 | 0% |
Name: Govern while LDAP is enabled behaves like registration and login allows the user to register and login File: register_spec.rb |
174 | 0 | 0% |
Executions table groupcompliance (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern Merge request approval with SAML SSO can be approved when requiring a password File: saml_sso_merge_request_approve_spec.rb |
175 | 0 | 0% |
Executions table groupthreat insights (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Govern Vulnerability report in a project can export vulnerability report to csv File: export_vulnerability_report_spec.rb |
551 | 0 | 0% |
Name: Govern vulnerability report with jira integration can successfully create a JIRA issue from vulnerability details page File: vulnerabilities_jira_integration_spec.rb |
174 | 0 | 0% |
Manage (27)
Executions table groupimport and integrate (27)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Manage Jenkins integration integrates and displays build status for MR pipeline in GitLab File: jenkins_build_status_spec.rb |
549 | 0 | 0% |
Name: Manage Gitlab migration with import within same instance successfully imports project File: gitlab_migration_project_spec.rb |
549 | 0 | 0% |
Name: Manage WebHooks integration sends a tag event File: webhook_events_spec.rb |
175 | 0 | 0% |
Name: Manage WebHooks integration sends an issues and note event File: webhook_events_spec.rb |
175 | 0 | 0% |
Name: Manage WebHooks integration sends a wiki page event File: webhook_events_spec.rb |
175 | 0 | 0% |
Name: Manage Pipeline status emails when pipeline fails sends an email File: pipeline_status_emails_spec.rb |
175 | 0 | 0% |
Name: Manage Pipeline status emails when pipeline passes sends an email File: pipeline_status_emails_spec.rb |
175 | 0 | 0% |
Name: Manage WebHooks integration sends a push event File: webhook_events_spec.rb |
175 | 0 | 0% |
Name: Manage WebHooks integration sends a merge request event File: webhook_events_spec.rb |
175 | 0 | 0% |
Name: Manage Group WebHooks integration sends subgroup events File: group_webhook_events_spec.rb |
175 | 0 | 0% |
Name: Manage Gitlab migration with release successfully imports project release File: gitlab_migration_release_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with project member member retains direct membership in imported project File: gitlab_migration_members_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with wiki successfully imports project wiki File: gitlab_migration_project_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with uninitialized project successfully imports project File: gitlab_migration_project_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with group member member retains indirect membership in imported project File: gitlab_migration_members_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with designs successfully imports design File: gitlab_migration_issue_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with associated merge request preserves related merge request File: gitlab_migration_issue_spec.rb |
174 | 0 | 0% |
Name: Manage GitHub import when imported via api imports repo push rules File: import_github_repo_spec.rb Exceptions: - Fabrication of QA::Resource::ProjectImportedFromGithub using the API failed (500) with '{"message":"500 Internal Server Error"}'.Correlation Id: 01HT9Z9FCFXZD00173WA52BGJZ
|
174 | 0 | 0% |
Name: Manage GitHub import when imported via api imports project File: import_github_repo_spec.rb Exceptions: - Fabrication of QA::Resource::ProjectImportedFromGithub using the API failed (500) with '{"message":"500 Internal Server Error"}'.Correlation Id: 01HT9Z9K60E1Y00CZBXG2W7DF6
|
174 | 0 | 0% |
Name: Manage Jira issues integration shows open issues File: jira_issues_list_spec.rb |
174 | 0 | 0% |
Name: Manage Jira integration closes an issue via pushing a commit File: jira_basic_integration_spec.rb |
174 | 0 | 0% |
Name: Manage Jira integration closes an issue via a merge request File: jira_basic_integration_spec.rb |
174 | 0 | 0% |
Name: Manage Jira issues integration searching issues returns results File: jira_issues_list_spec.rb |
174 | 0 | 0% |
Name: Manage Jira issues integration views an issue File: jira_issues_list_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with EE features imports group epics and iterations File: gitlab_migration_group_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with ci pipeline successfully imports ci pipeline File: gitlab_migration_pipeline_spec.rb |
174 | 0 | 0% |
Name: Manage Gitlab migration with project issues successfully imports issue File: gitlab_migration_issue_spec.rb |
174 | 0 | 0% |
Package (14)
Executions table grouppackage registry (4)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Package Maven group level endpoint duplication setting when disabled prevents users from publishing duplicates File: maven_group_level_spec.rb |
1082 | 0 | 0% |
Name: Package Maven group level endpoint via maven using a ci job token pushes and pulls a maven package File: maven_group_level_spec.rb |
1082 | 0 | 0% |
Name: Package Maven group level endpoint via maven using a personal access token pushes and pulls a maven package File: maven_group_level_spec.rb |
1082 | 0 | 0% |
Name: Package Maven group level endpoint via maven using a project deploy token pushes and pulls a maven package File: maven_group_level_spec.rb |
1082 | 0 | 0% |
Executions table groupcontainer registry (10)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Package Self-managed Container Registry when tls is disabled using docker:20.10.23 and a personal access token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:20.10.23 and a ci job token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:23.0.6 and a ci job token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:20.10.23 and a project deploy token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:24.0.1 and a personal access token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:23.0.6 and a personal access token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:23.0.6 and a project deploy token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:24.0.1 and a ci job token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Self-managed Container Registry when tls is disabled using docker:24.0.1 and a project deploy token pushes image and deletes tag File: container_registry_spec.rb |
350 | 0 | 0% |
Name: Package Dependency Proxy Group SSO pulls an image using the dependency proxy on a group enforced SSO File: dependency_proxy_sso_spec.rb |
175 | 0 | 0% |
Plan (8)
Executions table groupknowledge (5)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Testing wiki content manipulation inside a project by manipulating content on the page File: project_based_content_manipulation_spec.rb |
560 | 0 | 0% |
Name: Plan Testing wiki content manipulation inside a project by manipulating content on the page with spaces File: project_based_content_manipulation_spec.rb |
560 | 0 | 0% |
Name: Plan Testing wiki content manipulation inside a project by manipulating content on the page using git push File: project_based_content_manipulation_spec.rb |
560 | 0 | 0% |
Name: Plan Project Wiki with Wiki Sidebar has all expected links that work File: project_based_list_spec.rb |
558 | 0 | 0% |
Name: Plan Project Wiki with Wiki Page List has all expected links that work File: project_based_list_spec.rb |
558 | 0 | 0% |
Executions table groupproject management (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Custom email customizes email with additional text File: custom_email_spec.rb |
559 | 0 | 0% |
Name: Plan Issue closes via pushing a commit File: closes_issue_via_pushing_a_commit_spec.rb |
559 | 0 | 0% |
Executions table groupproduct planning (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Plan Epics milestone dates API updates epic dates when updating milestones File: epics_milestone_dates_spec.rb |
560 | 0 | 0% |
Systems (9)
Executions table groupgitaly (9)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Systems Changing Gitaly repository storage when moving from Gitaly Cluster to Gitaly behaves like repository storage move confirms a finished status after moving project repository storageFile: changing_repository_storage_spec.rb |
175 | 0 | 0% |
Name: Systems Changing Gitaly repository storage when moving from Gitaly to Gitaly Cluster behaves like repository storage move confirms a finished status after moving project repository storageFile: changing_repository_storage_spec.rb |
175 | 0 | 0% |
Name: Systems Changing Gitaly repository storage when moving from one Gitaly storage to another behaves like repository storage move confirms a finished status after moving project repository storageFile: changing_repository_storage_spec.rb |
175 | 0 | 0% |
Name: Systems Gitaly Cluster replication queue allows replication of different repository after interruption File: praefect_replication_queue_spec.rb |
171 | 0 | 0% |
Name: Systems Gitaly backend node recovery recovers from dataloss File: backend_node_recovery_spec.rb |
171 | 0 | 0% |
Name: Systems Gitaly distributed reads reads from each node File: distributed_reads_spec.rb |
171 | 0 | 0% |
Name: Systems Gitaly distributed reads when a node is unhealthy does not read from the unhealthy node File: distributed_reads_spec.rb |
171 | 0 | 0% |
Name: Systems with Gitaly automatic failover and recovery automatically fails over File: automatic_failover_and_recovery_spec.rb |
171 | 0 | 0% |
Name: Systems with Gitaly automatic failover and recovery when recovering from dataloss after failover automatically reconciles File: automatic_failover_and_recovery_spec.rb |
171 | 0 | 0% |
Verify (9)
Executions table grouppipeline authoring (5)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Update CI file with pipeline editor creates new pipelines, target branch, and merge request File: update_ci_file_with_pipeline_editor_spec.rb |
560 | 0 | 0% |
Name: Verify CI catalog when sorting with created at behaves like descending order by default displays from last to first File: ci_catalog_sorting_spec.rb |
550 | 0 | 0% |
Name: Verify CI catalog when sorting with created at behaves like ascending order displays from first to last File: ci_catalog_sorting_spec.rb |
550 | 0 | 0% |
Name: Verify CI catalog when sorting with released at behaves like descending order by default displays from last to first File: ci_catalog_sorting_spec.rb |
550 | 0 | 0% |
Name: Verify CI catalog when sorting with released at behaves like ascending order displays from first to last File: ci_catalog_sorting_spec.rb |
550 | 0 | 0% |
Executions table grouprunner (2)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Runner registration user registers a new project runner File: register_runner_spec.rb |
560 | 0 | 0% |
Name: Verify Runner fleet management shows group runner online count File: group_runner_status_counts_spec.rb |
558 | 0 | 0% |
Executions table grouppipeline security (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Project artifacts when user tries bulk deletion successfully delete them File: user_can_bulk_delete_artifacts_spec.rb |
559 | 0 | 0% |
Executions table grouppipeline execution (1)
NAME | RUNS | FAILURES | FAILURE RATE |
---|---|---|---|
Name: Verify Trigger child pipeline with 'when:manual' can trigger bridge job File: trigger_child_pipeline_with_manual_spec.rb |
557 | 0 | 0% |