Another transient failure in /qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb

Job: https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/378067348

There were two failures:

1st Failure

1st Try:

7207 1st Try error in ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:189:
7208 expected to find text "uses group managed accounts. You need to create a new GitLab account which will be managed by" in "Skip to content Projects Groups More S saml_sso_group-6f2c4211 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-6f2c4211 Details SAML for saml_sso_group-6f2c4211 was added to your connected accounts S saml_sso_group-6f2c4211 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created P project-in-saml-enforced-group-602e618c14824eb0 project in SAML enforced group for git clone test 0 34 seconds ago"

2nd Try:

7851   1) Manage Group SAML SSO Enforced SSO Group managed accounts removes existing users from the group, forces existing users to create a new account and allows to leave group
7852      Failure/Error: expect(page).to have_text("uses group managed accounts. You need to create a new GitLab account which will be managed by")
7853        expected to find text "uses group managed accounts. You need to create a new GitLab account which will be managed by" in "Skip to content Projects Groups More S saml_sso_group-6f2c4211 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-6f2c4211 Details Already signed in with SAML for saml_sso_group-6f2c4211 S saml_sso_group-6f2c4211 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created P project-in-saml-enforced-group-602e618c14824eb0 project in SAML enforced group for git clone test 0 54 seconds ago"
7854      # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:198:in `block (5 levels) in <module:QA>'

Screenshot for 1st failure 1st try:

Screenshot for 1st failure 2nd try:

2nd Failure:

1st Try:

7718 1st Try error in ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:24:
7719    expected to find text "SAML for saml_sso_group-6f2c4211 was added to your connected accounts" in "Skip to content Projects Groups More S saml_sso_group-6f2c4211 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-6f2c4211 Details SAML authentication failed: User has already been taken S saml_sso_group-6f2c4211 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2019-12-12-22-26-00-c04f496a3c74f087 Owner 0 2 1 S saml-sub-group Owner 0 0 1 P project-in-saml-enforced-group-for-access-test-b8fb155574d22d83 project in SAML enforced group for access test 0 30 seconds ago P project-in-saml-enforced-group-for-access-test-a196cb60a932ca39 project in SAML enforced group for access test 0 1 minute ago P project-in-saml-enforced-group-602e618c14824eb0 project in SAML enforced group for git clone test 0 2 minutes ago"
7720 ...or:
7721    expected to find text "Already signed in with SAML for saml_sso_group-6f2c4211" in "Skip to content Projects Groups More S saml_sso_group-6f2c4211 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-6f2c4211 Details SAML authentication failed: User has already been taken S saml_sso_group-6f2c4211 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2019-12-12-22-26-00-c04f496a3c74f087 Owner 0 2 1 S saml-sub-group Owner 0 0 1 P project-in-saml-enforced-group-for-access-test-b8fb155574d22d83 project in SAML enforced group for access test 0 30 seconds ago P project-in-saml-enforced-group-for-access-test-a196cb60a932ca39 project in SAML enforced group for access test 0 1 minute ago P project-in-saml-enforced-group-602e618c14824eb0 project in SAML enf

2nd Try:

7874   2) Manage Group SAML SSO Non enforced SSO User logs in to group with SAML SSO
7875      Failure/Error:
7876        expect(page).to have_content("SAML for #{Runtime::Env.sandbox_name} was added to your connected accounts")
7877                          .or have_content("Already signed in with SAML for #{Runtime::Env.sandbox_name}")
7878      
7879           expected to find text "SAML for saml_sso_group-6f2c4211 was added to your connected accounts" in "Skip to content Projects Groups More S saml_sso_group-6f2c4211 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-6f2c4211 Details SAML authentication failed: User has already been taken S saml_sso_group-6f2c4211 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2019-12-12-22-26-00-c04f496a3c74f087 Owner 0 2 1 S saml-sub-group Owner 0 0 1 P project-in-saml-enforced-group-for-access-test-b8fb155574d22d83 project in SAML enforced group for access test 0 47 seconds ago P project-in-saml-enforced-group-for-access-test-a196cb60a932ca39 project in SAML enforced group for access test 0 1 minute ago P project-in-saml-enforced-group-602e618c14824eb0 project in SAML enforced group for git clone test 0 2 minutes ago"
7880      
7881        ...or:
7882      
7883           expected to find text "Already signed in with SAML for saml_sso_group-6f2c4211" in "Skip to content Projects Groups More S saml_sso_group-6f2c4211 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-6f2c4211 Details SAML authentication failed: User has already been taken S saml_sso_group-6f2c4211 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2019-12-12-22-26-00-c04f496a3c74f087 Owner 0 2 1 S saml-sub-group Owner 0 0 1 P project-in-saml-enforced-group-for-access-test-b8fb155574d22d83 project in SAML enforced group for access test 0 47 seconds ago P project-in-saml-enforced-group-for-access-test-a196cb60a932ca39 project in SAML enforced group for access test 0 1 minute ago P project-in-saml-enforced-group-602e618c14824eb0 project in SAML enforced group for git clone test 0 2 minutes ago"
7884      # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:257:in `login_to_idp_if_required_and_expect_success'
7885      # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:39:in `block (4 levels) in <module:QA>'

Screenshot for 2nd failure 1st try:

Screenshot for 2nd failure 2nd try:

added devopsmanage found:in MR groupauthentication and authorization [DEPRECATED] priority1 severity1 typebug + 1 deleted label

Another transient failure in the same test reported earlier: #39607 (closed)

mentioned in issue #39607 (closed)

mentioned in commit a3ed21ad

mentioned in issue #103397 (closed)

Occurrences:

• https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/383230843
• https://gitlab.com/gitlab-org/quality/nightly/-/jobs/390578074

added found:staging-orchestrated label

mentioned in issue gitlab-org/quality/pipeline-triage#8 (closed)

mentioned in issue #121479 (closed)

mentioned in issue #193026 (closed)

added found:nightly label

mentioned in commit d58847b7

Reproduced the issue locally. The problem appears to be that group manage accounts are not getting turned on correctly. A possible fix is to wait between retries to give enough time for animation to complete.

mentioned in merge request !22360 (merged)

mentioned in commit 71d6a839

added workflowin review label

mentioned in issue #195098 (closed)

closed via merge request !22360 (merged)

mentioned in commit 7fea5e9f

Re-occured: https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/395399648

1st failure 1st try:

6437 1st Try error in ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:189:
6438 expected to find text "uses group managed accounts. You need to create a new GitLab account which will be managed by" in "Skip to content Projects Groups More S saml_sso_group-196a0438 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-196a0438 Details SAML for saml_sso_group-196a0438 was added to your connected accounts S saml_sso_group-196a0438 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2020-01-07-07-21-44-a0ebd6fbfb48e628 0 2 0 S saml-sub-group 0 0 0 P project-in-saml-enforced-group-for-access-test-be58a298682d21d1 project in SAML enforced group for access test 0 53 seconds ago P project-in-saml-enforced-group-for-access-test-82b3e193496b1b0a project in SAML enforced group for access test 0 1 minute ago P project-in-saml-enforced-group-e5ad52edb4c47204 project in SAML enforced group for git clone test 0 1 minute ago"

1st failure 2nd try:

7103   1) Manage Group SAML SSO Enforced SSO Group managed accounts removes existing users from the group, forces existing users to create a new account and allows to leave group
7104      Failure/Error: expect(page).to have_text("Sign up was successful! Please confirm your email to sign in.")
7105        expected to find text "Sign up was successful! Please confirm your email to sign in." in "saml_sso_group-196a0438 uses group managed accounts. You need to create a new GitLab account which will be managed by saml_sso_group-196a0438. GitLab Enterprise Edition Open source software to collaborate on code Manage Git repositories with fine-grained access controls that keep your code secure. Perform code reviews and enhance collaboration with merge requests. Each project can also have an issue tracker and a wiki. S Finish setting up your dedicated account for . Email Full name Username You'll be signed out from your current account automatically. Administrator @root admin@example.com Explore Help About GitLab"
7106      # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:208:in `block (5 levels) in <module:QA>'

Screenshot for 1st failure 1st try:

Screenshot for 1st failure 2nd try:

2nd failure 1st try:

6871 1st Try error in ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:24:
6872    expected to find text "SAML for saml_sso_group-196a0438 was added to your connected accounts" in "Skip to content Projects Groups More S saml_sso_group-196a0438 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-196a0438 Details SAML authentication failed: User has already been taken S saml_sso_group-196a0438 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2020-01-07-07-21-44-a0ebd6fbfb48e628 0 2 0 S saml-sub-group 0 0 0 P project-in-saml-enforced-group-for-access-test-be58a298682d21d1 project in SAML enforced group for access test 0 1 minute ago P project-in-saml-enforced-group-for-access-test-82b3e193496b1b0a project in SAML enforced group for access test 0 2 minutes ago P project-in-saml-enforced-group-e5ad52edb4c47204 project in SAML enforced group for git clone test 0 2 minutes ago"
6873 ...or:
6874    expected to find text "Already signed in with SAML for saml_sso_group-196a0438" in "Skip to content Projects Groups More S saml_sso_group-196a0438 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-196a0438 Details SAML authentication failed: User has already been taken S saml_sso_group-196a0438 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2020-01-07-07-21-44-a0ebd6fbfb48e628 0 2 0 S saml-sub-group 0 0 0 P project-in-saml-enforced-group-for-access-test-be58a298682d21d1 project in SAML enforced group for access test 0 1 minute ago P project-in-saml-enforced-group-for-access-test-82b3e193496b1b0a project in SAML enforced group for access test 0 2 minutes ago P project-in-saml-enforced-group-e5ad52edb4c47204 project in SAML enforced group for git clone test 0 2 minutes ago"

2nd failure 2nd try:

7126   2) Manage Group SAML SSO Non enforced SSO User logs in to group with SAML SSO
7127      Failure/Error:
7128        expect(page).to have_content("SAML for #{Runtime::Env.sandbox_name} was added to your connected accounts")
7129                          .or have_content("Already signed in with SAML for #{Runtime::Env.sandbox_name}")
7130      
7131           expected to find text "SAML for saml_sso_group-196a0438 was added to your connected accounts" in "Skip to content Projects Groups More S saml_sso_group-196a0438 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-196a0438 Details SAML authentication failed: User has already been taken S saml_sso_group-196a0438 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2020-01-07-07-21-44-a0ebd6fbfb48e628 0 2 0 S saml-sub-group 0 0 0 P project-in-saml-enforced-group-for-access-test-be58a298682d21d1 project in SAML enforced group for access test 0 2 minutes ago P project-in-saml-enforced-group-for-access-test-82b3e193496b1b0a project in SAML enforced group for access test 0 2 minutes ago P project-in-saml-enforced-group-e5ad52edb4c47204 project in SAML enforced group for git clone test 0 3 minutes ago"
7132      
7133        ...or:
7134      
7135           expected to find text "Already signed in with SAML for saml_sso_group-196a0438" in "Skip to content Projects Groups More S saml_sso_group-196a0438 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-196a0438 Details SAML authentication failed: User has already been taken S saml_sso_group-196a0438 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created Q qa-test-2020-01-07-07-21-44-a0ebd6fbfb48e628 0 2 0 S saml-sub-group 0 0 0 P project-in-saml-enforced-group-for-access-test-be58a298682d21d1 project in SAML enforced group for access test 0 2 minutes ago P project-in-saml-enforced-group-for-access-test-82b3e193496b1b0a project in SAML enforced group for access test 0 2 minutes ago P project-in-saml-enforced-group-e5ad52edb4c47204 project in SAML enforced group for git clone test 0 3 minutes ago"
7136      # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:257:in `login_to_idp_if_required_and_expect_success'
7137      # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:39:in `block (4 levels) in <module:QA>'

Screenshot for 2nd failure 1st try:

Screenshot for 2nd failure second try:

assigned to @sliaquat

reopened

changed the description

Setting ~"Category:Authentication and Authorization" based on ~"group::access".

Setting ~"Category:Authentication and Authorization" based on ~"group::access".

Another failure: https://gitlab.com/gitlab-org/quality/nightly/-/jobs/396479208 but this time it was just one test failing:

1st try:

6471 1st Try error in ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:159:
6472 Unable to find css "[data-qa-selector=\"enforced_sso_toggle_button\"],.qa-enforced-sso-toggle-button"

Expand for screenshot

2nd try:

7600   1) Manage Group SAML SSO Enforced SSO Group managed accounts removes existing users from the group, forces existing users to create a new account and allows to leave group
7601      Failure/Error: expect(page).to have_text("uses group managed accounts. You need to create a new GitLab account which will be managed by")
7602        expected to find text "uses group managed accounts. You need to create a new GitLab account which will be managed by" in "Skip to content Projects Groups More S saml_sso_group-4ea046e8 Group overview Details Activity Contribution Analytics Insights Epics 0 Issues 0 Merge Requests 0 Security Kubernetes Packages Members Settings Collapse sidebar saml_sso_group-4ea046e8 Details SAML authentication failed: User has already been taken S saml_sso_group-4ea046e8 Group ID: 2 Subgroups and projects Shared projects Archived projects Last created P project-in-saml-enforced-group-4a039c638effea02 project in SAML enforced group for git clone test 0 52 seconds ago"
7603      # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:198:in `block (5 levels) in <module:QA>'

Expand for screenshot

Happened here as well: https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/397716070

mentioned in issue gitlab-org/quality/pipeline-triage#10 (closed)

changed due date to January 23, 2020

changed milestone to %12.8

added priority2 label and removed priority1 label

mentioned in merge request !22700 (merged)

added QA Quality labels

/cc @dchevalier2 since you're devopsmanage counterpart.

Setting ~"Category:Authentication and Authorization" based on ~"group::access".

added 1 deleted label

Another (slight different) failure: https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/398719528

8146 An error occurred in an `after(:context)` hook.
8147 Failure/Error: saml_sso.click_save_changes
8148 Capybara::ElementNotFound:
8149   Unable to find css "[data-qa-selector=\"save_changes_button\"],.qa-save-changes-button"
8150 # /usr/local/bundle/gems/capybara-3.29.0/lib/capybara/node/finders.rb:296:in `block in synced_resolve'
8151 # /usr/local/bundle/gems/capybara-3.29.0/lib/capybara/node/base.rb:83:in `synchronize'
8152 # /usr/local/bundle/gems/capybara-3.29.0/lib/capybara/node/finders.rb:287:in `synced_resolve'
8153 # /usr/local/bundle/gems/capybara-3.29.0/lib/capybara/node/finders.rb:52:in `find'
8154 # /usr/local/bundle/gems/capybara-3.29.0/lib/capybara/session.rb:749:in `block (2 levels) in <class:Session>'
8155 # /usr/local/bundle/gems/capybara-3.29.0/lib/capybara/dsl.rb:51:in `block (2 levels) in <module:DSL>'
8156 # ./qa/page/base.rb:84:in `find_element'
8157 # ./qa/support/page/logging.rb:45:in `find_element'
8158 # ./qa/page/base.rb:115:in `click_element'
8159 # ./qa/support/page/logging.rb:81:in `click_element'
8160 # ./qa/ee/page/group/settings/saml_sso.rb:62:in `click_save_changes'
8161 # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:368:in `block in disable_enforce_sso_and_group_managed_account'
8162 # ./qa/scenario/actable.rb:16:in `perform'
8163 # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:364:in `disable_enforce_sso_and_group_managed_account'
8164 # ./qa/specs/features/ee/browser_ui/1_manage/group/group_saml_sso_spec.rb:237:in `block (4 levels) in <module:QA>'

got this one here ^, too: https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/398735045

This also causes other quarantine jobs to fail, e.g., this elasticsearch job: https://gitlab.com/gitlab-org/gitlab-qa/-/jobs/398816921

Looking into it.

This particular issue should be fixed with !22751 (merged)

mentioned in commit 4f732a57

mentioned in merge request !21984 (merged)

mentioned in merge request !22751 (merged)

mentioned in merge request !22759 (closed)

mentioned in commit 86f15146

mentioned in commit 7816adee

mentioned in merge request !17139 (merged)

mentioned in merge request !21721 (merged)

mentioned in issue gitlab-org/quality/toolbox#4 (closed)

mentioned in merge request !24172 (merged)

closed via merge request !24172 (merged)

mentioned in commit da4917c9

added failureflaky-test label