Above all else, please remember that the aim of pipeline triage is to identify problems and try to get them resolved (ideally) before they impact users.
Failure in browser_ui/5_package/container_registry/container_registry_omnibus_spec.rb | Package Self-managed Container Registry when tls is disabled using docker:20.10 and a personal access token pushes image and deletes tag
Failure in browser_ui/4_verify/user_cancel_merge_request_in_merge_train_spec.rb | Verify Cancelling merge request in merge train when user cancels the merge request does not create a TODO task
Failure in browser_ui/3_create/web_ide_new/add_new_directory_in_web_ide_spec.rb | Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed
Failure in browser_ui/8_monitor/alert_management/alert_with_escalation_policy_spec.rb | Monitor Alert with escalation policy notifies on-call user via system note and email on new alert
browser_ui/8_monitor/alert_management/alert_with_escalation_policy_spec.rb has been failing for a while and keeps creating new issues because of the changing error message. Unfortunately I didn't get a chance to look into it today.
browser_ui/8_monitor/alert_management/alert_with_escalation_policy_spec.rb has been failing for a while and keeps creating new issues because of the changing error message.
Will keep an eye on it, so far failure didn't happen
Nightlies are failing because of a security release (this is expected - but maybe we could do better, e.g., automatically suspend the nightlies? )
IMO it's better to simply move all tests in to gitlab project and just run via the standard nightly setup using common setup. Then the tests won't be affected by security releases and it will also be somewhat easier to track when issues got introduced since we will have actual commit diffs.
I don't believe testing against nightly build specifically has much additional benefit.
Failure in browser_ui/3_create/web_ide_new/add_new_directory_in_web_ide_spec.rb | Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed
Failure in browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb | Create Restricted protected branch push and merge when only one user is allowed to merge and push to a protected branch behaves like selected developer user pushes and m...
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging several times behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in browser_ui/5_package/container_registry/container_registry_omnibus_spec.rb | Package Self-managed Container Registry when tls is disabled using docker:20.10 and a personal access token pushes image and deletes tag
Failure in api/3_create/repository/project_archive_compare_spec.rb | Create Compare archives of different user projects with the same name and check they're different download archives of each user project then check they are different
Failure in browser_ui/9_data_stores/elasticsearch/elasticsearch_reindexing_spec.rb | Data Stores Search using Elasticsearch tests reindexing after webIDE
Failure in api/1_manage/migration/gitlab_migration_project_spec.rb | Manage Gitlab migration with import within same instance successfully imports project
Failure in api/1_manage/project_access_token_spec.rb | Manage Project access token for the same project can be used to create a file via the project API
Failure in browser_ui/3_create/web_ide_new/add_new_directory_in_web_ide_spec.rb | Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed
Failure in browser_ui/5_package/container_registry/container_registry_omnibus_spec.rb | Package Self-managed Container Registry when tls is disabled using docker:20.10 and a personal access token pushes image and deletes tag
Failure in browser_ui/13_secure/license_compliance_spec.rb | Secure License Compliance when a CycloneDX SBOM file exists License Management behaves like approve license can approve a license in the settings page
Based on recent discussions, this bug should be resolved by https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/5899, but that is currently scheduled for %15.11. It is the end of my day here, but I'll reach out to the team tomorrow to see if we can get eyes on that sooner or if we will need to quarantine or adjust these tests in the meantime
Failure in browser_ui/13_secure/merge_request_license_widget_spec.rb:231 | Secure License merge request widget when a CycloneDX SBOM file exists manage licenses from the merge requestfai
Failure in browser_ui/5_package/container_registry/container_registry_omnibus_spec.rb | Package Self-managed Container Registry when tls is disabled using docker:20.10 and a personal access token pushes image and deletes tag
Failure in browser_ui/3_create/web_ide_new/add_new_directory_in_web_ide_spec.rb | Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed
Failure in api/1_manage/migration/gitlab_migration_project_spec.rb | Manage Gitlab migration with import within same instance successfully imports project
Failure in browser_ui/11_fulfillment/saas_user_limit_experience_spec.rb | Fulfillment Utilization when Saas user limit experience feature flags are enabled limit overage enforcement removed from private group when trial is started
Failure in browser_ui/13_secure/license_compliance_spec.rb | Secure License Compliance when a CycloneDX SBOM file exists License Management behaves like approve license can approve a license in the settings page
Failure in api/1_manage/migration/gitlab_migration_project_spec.rb | Manage Gitlab migration with import within same instance successfully imports project
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging once behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in browser_ui/3_create/web_ide_new/add_new_directory_in_web_ide_spec.rb | Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed
Started to happen towards the end of my day again. @rodrigo.tomonari has been helping from the import team looking into this. Tried to assist as much as I could while still online, but if someone is able to help follow-up while I'm offline I'd greatly appreciate it Slack convo here: https://gitlab.slack.com/archives/CLX7WMSKW/p1679954026353529
Started happening towards the end of my day as well, caused by a handful of user cap feature flags being enabled in Staging. I notified @dstull, who is looking into a fix for the tests
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging several times behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in browser_ui/13_secure/merge_request_license_widget_spec.rb | Secure License merge request widget when a CycloneDX SBOM file exists manage licenses from the merge request
Failure in api/1_manage/migration/gitlab_migration_project_spec.rb | Manage Gitlab migration with import within same instance successfully imports project
Failure in api/1_manage/migration/gitlab_migration_project_spec.rb | Manage Gitlab migration with import within same instance successfully imports project
Failure in browser_ui/11_fulfillment/saas_user_limit_experience_spec.rb | Fulfillment Utilization when Saas user limit experience feature flags are enabled limit overage enforcement removed from private group when trial is started
Failure in browser_ui/13_secure/license_compliance_spec.rb | Secure License Compliance when a CycloneDX SBOM file exists License Management behaves like approve license can approve a license in the settings page
Failure in browser_ui/13_secure/merge_request_license_widget_spec.rb | Secure License merge request widget when a CycloneDX SBOM file exists manage licenses from the merge request
Failure in browser_ui/13_secure/license_compliance_spec.rb | Secure License Compliance when a CycloneDX SBOM file exists License Management behaves like approve license can approve a license in the settings page
Failure in browser_ui/3_create/repository/restrict_push_protected_branch_spec.rb | Create Restricted protected branch push and merge when only one user is allowed to merge and push to a protected branch behaves like selected developer user pushes and m...
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging several times behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in /spec/ui/self_managed/renew_license_spec.rb | User renews self-managed subscription with cloud license successfully renewed with true-up users
Failure in browser_ui/11_fulfillment/saas_user_limit_experience_spec.rb | Fulfillment Utilization when Saas user limit experience feature flags are enabled limit overage enforcement removed from private group when trial is started
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging once behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in browser_ui/10_govern/scan_result_policy_vulnerabilities_spec.rb | Govern Scan result policy requires approval when a pipeline report has findings matching the scan result policy
Failure in browser_ui/4_verify/ci_variable/pipeline_with_protected_variable_spec.rb | Verify Pipeline with protected variable does not expose variable on unprotected branch
Failure in api/1_manage/user_access_termination_spec.rb | Manage User after parent group membership termination is not allowed to push code via the CLI
Failure in browser_ui/4_verify/ci_job_artifacts/unlocking_job_artifacts_across_pipelines_spec.rb | Verify Unlocking job artifacts across pipelines when latest pipeline failed keeps job artifacts from latest failed pipelines and from latest successful p...
Failure in browser_ui/3_create/merge_request/create_merge_request_from_push_notification_spec.rb | Create Create a new merge request from the event notification after a push creates a merge request after a push via the API
Failure in browser_ui/7_configure/auto_devops/auto_devops_templates_spec.rb | Configure AutoDevOps Templates using express template works with Auto DevOps
Several tests have failed with Resource at https://gitlab.com/api/v4/runners/:id?private_token=[****] could not be found (403): `{"message":"403 Forbidden - No access granted"}`.
I started doing some investigation (gitlab-org/gitlab#403174 (comment 1334144010)) but kept getting interrupted by all those other failures (and the ones I listed today are again just the ones that haven't been reported previously this week).
Failure in browser_ui/4_verify/ci_job_artifacts/unlocking_job_artifacts_across_pipelines_spec.rb | Verify Unlocking job artifacts across pipelines when latest pipeline failed keeps job artifacts from latest failed pipelines and from latest successful p...
Failure in browser_ui/11_fulfillment/saas_user_limit_experience_spec.rb | Fulfillment Utilization when Saas user limit experience feature flags are enabled limit overage enforcement removed from private group when trial is started
Failure in browser_ui/13_secure/license_compliance_spec.rb | Secure License Compliance when a CycloneDX SBOM file exists License Management behaves like approve license can approve a license in the settings page
Failure in browser_ui/13_secure/license_compliance_spec.rb | Secure License Compliance when a CycloneDX SBOM file exists License Management behaves like approve license can approve a license in the settings page
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging several times behaves like merge when pipeline succeeds merges after pipeline succeeds
Failures in multiple E2E tests | api/v4/runners... (gitlab-org/gitlab#403174 - closed) - please continue to investigate this issue. Unfortunately Staging Ref debugging took too much time and I hasn't been able to dive into this, but looks like it's now on Pre as well - which might indicate that it's some runner issue? maybe a new version was released recently?
Failures in multiple E2E tests | api/v4/runners... (gitlab-org/gitlab#403174 - closed) - please continue to investigate this issue. Unfortunately Staging Ref debugging took too much time and I hasn't been able to dive into this, but looks like it's now on Pre as well - which might indicate that it's some runner issue? maybe a new version was released recently?
I left a comment with some additional findings that could possibly be related to project auth update delays again. I also reached out to @manojmj to see if he may be able to assist in investigating as well.
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging once behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in browser_ui/3_create/repository/branch_with_unusual_name_spec.rb | Create Branch with unusual name when branch name contains slash, hash, double dash, and capital letter renders repository file tree correctly
Failure in browser_ui/3_create/repository/merge_with_code_owner_in_root_group_spec.rb | Create Codeowners when the project is in the root group and the code owner is the root group behaves like code owner merge request is approved and merged
Failure in browser_ui/15_growth/free_trial_spec.rb | Growth SaaS trials starts a free trial when on about page with multiple eligible namespaces registers for a new trial
Failure in browser_ui/13_secure/merge_request_license_widget_spec.rb | Secure License merge request widget when a CycloneDX SBOM file exists manage licenses from the merge request
Failure in browser_ui/11_fulfillment/purchase/overage_modal_spec.rb | Fulfillment Seat overage modal with ultimate plan for group invite does not show overage modal when inviting a group which does not increase seats owed
Failure in browser_ui/11_fulfillment/purchase/overage_modal_spec.rb | Fulfillment Seat overage modal with ultimate plan for group invite when inviting a group with developer role which increases seats owed behaves like overage for group invite shows th...
Failure in browser_ui/11_fulfillment/saas_user_limit_experience_spec.rb | Fulfillment Utilization when Saas user limit experience feature flags are enabled preview notification displayed for private group when over limit
Failure in browser_ui/11_fulfillment/saas_user_limit_experience_spec.rb | Fulfillment Utilization when Saas user limit experience feature flags are enabled enforcement limit counts includes invited group and project members
merge_with_code_owner_in_root_group_spec fails with a verification error, which is clearly related to the feature flags. But it seems the other tests fail when trying to create a second group. The tests create the first group and then expect the browser to be at the dashboard, but for some reason it's still at the page showing the first group's details, so when it tries to create another group, it fails because that menu item is found on the dashboard.
merge_with_code_owner_in_root_group_spec passed after identity_verification_phone_number and soft_email_confirmation were disabled, but the other tests still fail. It's strange that they would still fail even though arkose_labs_signup_challenge and identity_verification were already enabled on staging.
Oh, almost forgot! I think I know how to fix unlocking_job_artifacts_across_parent_child_pipelines_spec but didn't have time to try it: gitlab-org/gitlab#403391 (comment 1336075752)
Failure in browser_ui/3_create/repository/branch_with_unusual_name_spec.rb | Create Branch with unusual name when branch name contains slash, hash, double dash, and capital letter renders repository file tree correctly
Failure in browser_ui/8_monitor/alert_management/recovery_alert_resolves_correct_alert_spec.rb | Monitor Recovery alert when using HTTP endpoint integration behaves like triggers recovery alert only resolves the correct alert
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging several times behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in browser_ui/4_verify/user_cancel_merge_request_in_merge_train_spec.rb | Verify Cancelling merge request in merge train when user cancels the merge request does not create a TODO task
Failure in browser_ui/8_monitor/incident_management/recovery_alert_closes_correct_incident_spec.rb | Monitor Recovery alert when using HTTP endpoint integration behaves like triggers recovery alert only closes the correct incident
Failure in browser_ui/3_create/web_ide_new/add_new_directory_in_web_ide_spec.rb | Create Add a directory in Web IDE when a directory with the same name already exists throws an error
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging several times behaves like merge when pipeline succeeds merges after pipeline succeeds
Failure in browser_ui/13_secure/merge_request_license_widget_spec.rb:231 | Secure License merge request widget when a CycloneDX SBOM file exists manage licenses from the merge requestfai
investigated few failures from Nightly locally, so far not sure why it's failing but then it turned out that it was issue gitlab-org/gitlab#395512 (closed)
Failure in browser_ui/3_create/repository/branch_with_unusual_name_spec.rb | Create Branch with unusual name when branch name contains slash, hash, double dash, and capital letter renders repository file tree correctly
Failure in browser_ui/3_create/merge_request/merge_when_pipeline_succeeds_spec.rb | Create Merge requests when merging several times behaves like merge when pipeline succeeds merges after pipeline succeeds