Page error checker fails test if group name contains 500
<!--- Before opening a new QA failure issue, make sure to first search for it in the QA failures board: https://gitlab.com/groups/gitlab-org/-/boards/1385578 The issue should have the following: - The relative path of the failing spec file in the title, e.g. if the login test fails, include `qa/specs/features/browser_ui/1_manage/login/log_in_spec.rb` in the title. This is required so that existing issues can easily be found by searching for the spec file. - If the issue is about multiple test failures, include the path for each failing spec file in the description. - A link to the failing job. - The stack trace from the job's logs in the "Stack trace" section below. - A screenshot (if available), and HTML capture (if available), in the "Screenshot / HTML page" section below. - A link to the corresponding test case(s) in the summary. ---> ### Summary An edge case was discovered where a group name with a uuid contained the text '500' and triggered the page error checker to fail the tests associated with this group the error checker checks for the first header text to include a 500, and in this case the group's uuid happened to. ``` five_hundred_test = Nokogiri::HTML.parse(page.html).xpath("//h1/text()").map.first unless five_hundred_test.nil? error_code = 500 if five_hundred_test.text.include?('500') end ``` Failing job(s): https://ops.gitlab.net/gitlab-org/quality/staging-canary/-/jobs/7192174 Failing spec(s): Corresponding test case(s): ### Stack trace ``` 6) Package Maven group level endpoint duplication setting when disabled using a personal access token prevents users from publishing group level Maven packages duplicates Failure/Error: package_project.group.visit! RuntimeError: Error Code 500 Status code 500 found Path: /gitlab-qa-sandbox-group/qa-test-2022-05-25-12-12-16-d4500c2e79c37289 ``` ### Screenshot / HTML page ![image](/uploads/d1bc1e9682116b47c9acf17d1b75d29b/image.png) ![Screen_Shot_2022-05-25_at_9.26.39_AM](/uploads/a12d3620b722c8f0d232659c5c58d769/Screen_Shot_2022-05-25_at_9.26.39_AM.png) <!-- Attach the screenshot and HTML snapshot of the page from the job's artifacts: 1. Download the job's artifacts and unarchive them. 1. Open the `gitlab-qa-run-2020-*/gitlab-{ce,ee}-qa-*/{,ee}/{api,browser_ui}/<path to failed test>` folder. 1. Select the `.png` and `.html` files that appears in the job logs (look for `HTML screenshot: /path/to/html/page.html` / `Image screenshot: `/path/to/html/page.png`). 1. Drag and drop them here. Note: You don't need to include a screenshot if the information it contains can be included as text. Include the text instead. E.g., error 500/404, "Retry later" errors, etc. If you include multiple screenshots it can be helpful to hide all but the first in a details/summary element, to avoid excessive scrolling: <details><summary>Expand for screenshot</summary> drag and drop the screenshot here </details> --> ### Possible fixes It could be worth looking into the possibility of a more specific selector for capturing page level errors. <!-- Default due date. --> <!-- Base labels. --> <!-- Test failure type label, please use just one.--> <!-- Choose the stage that appears in the test path, e.g. ~"devops::create" for `qa/specs/features/browser_ui/3_create/web_ide/add_file_template_spec.rb`. --> <!-- Select a label for where the failure was found, e.g. if the failure occurred in a nightly pipeline, select ~"found:nightly". --> <!-- https://about.gitlab.com/handbook/engineering/quality/guidelines/#priorities: - ~"priority::1": Tests that are needed to verify fundamental GitLab functionality. - ~"priority::2": Tests that deal with external integrations which may take a longer time to debug and fix. --> <!-- Select the current milestone if ~"priority::1" or the next milestone if ~"priority::2". -->
issue