Failure in browser_ui/secure/security_reports_spec.rb | Secure Security Reports displays the Dependency List
Full description
Secure Security Reports displays the Dependency List
File path
./qa/specs/features/ee/browser_ui/secure/security_reports_spec.rb
Stack trace
1) Secure Security Reports populated reports are displayed displays the Dependency List
Failure/Error: expect(dependency_list).to have_dependency_count_of number_of_dependencies_in_fixture
expected #has_dependency_count_of?(9) to return true, got false
# ./qa/specs/features/ee/browser_ui/secure/security_reports_spec.rb:150:in `block (5 levels) in <module:QA>'
# ./qa/scenario/actable.rb:16:in `perform'
# ./qa/specs/features/ee/browser_ui/secure/security_reports_spec.rb:149:in `block (4 levels) in <module:QA>'First happened in https://gitlab.com/gitlab-org/gitlab-qa-mirror/-/jobs/1184904814.
Implementation plan
This issue may have been fixed by Ensure dependency list shows successful pipelines, please see this comment for more details
-
Ensure that the tests quarantined in this MR no longer fail in the nightly build since
July 12 2021 12:20PM EST -
Un-quarantine the tests listed in step
1. - Ensure that the above tests no longer fail
Activity
added Enterprise Edition QA Quality failureinvestigating found:master priority2 test labels
marked this issue as related to gitlab-org/quality/testcases#564 (closed)
added devopssecure groupcomposition analysis labels
Happened again on nightly - https://gitlab.com/gitlab-org/quality/nightly/-/jobs/1189263662
Screenshot shows only 7 dependencies (out of expected 9):
/cc @willmeek
Can see there is already a bug report open - #328059 (closed), will link it to this issue.
I am going to quarantine the test on master and nightly only till this is fixed - !59710 (merged) as it popped up in several MRs I was reviewing and creates quite a lot of noise for pipeline triage in general.
marked this issue as related to #328059 (closed)
added typebug label and removed failureinvestigating label
mentioned in issue gitlab-org/quality/pipeline-triage#78 (closed)
added sectionsec label
mentioned in commit a6b12d5b
mentioned in commit omnibus-gitlab@f2c23710
mentioned in commit omnibus-gitlab@a322670d
added severity2 label
changed milestone to %Next 1-3 releases
mentioned in commit 089e8af7
added [deprecated] Accepting merge requests label
mentioned in merge request !59710 (merged)
mentioned in issue gitlab-org/quality/pipeline-triage#80 (closed)
changed milestone to %14.2
added found:main label
mentioned in commit 287e66c4
mentioned in commit 68be0c5c
mentioned in issue gitlab-org/quality/pipeline-triage#84 (closed)
mentioned in merge request !62607 (merged)
mentioned in merge request !61888 (merged)
mentioned in merge request !62689 (merged)
This groupcomposition analysis bug has at most 25% of the SLO duration remaining and is ~"approaching-SLO" breach. Please consider taking action before this becomes a ~"missed-SLO" in 14 days (2021-06-18).
added SLONear Miss label
added backend label
added GitLab Ultimate label
added Category:Dependency Scanning [DEPRECATED] dependency list labels
added SLOMissed label
added workflowplanning breakdown label
added workflowrefinement label and removed workflowplanning breakdown label
changed milestone to %14.3
changed milestone to %Backlog
This occurred again on Canary - https://ops.gitlab.net/gitlab-org/quality/canary/-/jobs/4510157 The screenshot indicates the same result (7 instead of 9) as in my comment above.
mentioned in issue gitlab-org/quality/pipeline-triage#93 (closed)
changed milestone to %14.3
@willmeek @a_mcdonald I believe this issue has been fixed by Ensure dependency list shows successful pipelines and we can now un-quarantine these tests.
Here's a video showing the behaviour before the code change in Ensure dependency list shows successful pipelines
And here's a video showing the behaviour after the code change:
I believe this issue has been fixed by Ensure dependency list shows successful pipelines and we can now un-quarantine these tests.
@a_mcdonald actually, I just noticed the date on your comment which showed a failure in Canary was
Aug 5, 2021 7:36am EST. Meanwhile, the code update that I thought might have fixed this issue: Ensure dependency list shows successful pipelines, was pushed to Canary onJul 12, 2021 7:50am EST, so it looks like my assumption is incorrect and this failing test has not been resolved by the code change.@willmeek any idea why we're still seeing this failure in canary given the video I shared in this comment which shows that the dependency list is only updated after a successful pipeline?
Hi Adam, I've tried to replicate it but been unable to. The latest Canary runs show the test as working. Unfortunately the test case issue gitlab-org/quality/testcases#564 (closed) won't show all updates from the QA bot though, which would list all failures.
I'll try and dig in if there were Canary issues on the 5th August that might be relevant.
@willmeek I didn't notice that you had already de-quarantined the security report spec tests last week, and it looks as if the tests have been passing consistently, so it seems as though ensure dependency list shows successful pipelines has actually solved the issue and that the Canary failure on August 5th may be a red-herring. What do you think about closing this issue and re-opening if we encounter another failure? /cc @gonzoyumo
@adamcohen I'll close the issue and if the same thing is observed again we can reopen and investigate then. Thank you!!
assigned to @adamcohen
removed [deprecated] Accepting merge requests label
added workflowscheduling label and removed workflowrefinement label
unassigned @adamcohen
added workflowrefinement label and removed workflowscheduling label
assigned to @adamcohen
added Stretch label
