UX/UI considerations for VET False Positives

Problem to solve

With the new creation of VET's False positive label, we need to consider how we should handle vulnerability findings with the False positive flag in the MR.

@andyvolpe and I hypothesize that security pros would want to still manually verify that it is, indeed, a FP, and that it would still be valuable to bring these vulnerability findings into the Vulnerability Report. It MIGHT be the case that they want to verify and then if they agree with the logic in identifying FPs, i.e. they build trust with the tool, they may want to ability to hide from the MR or even auto-dismiss them later on.

Considerations:

  • Show/ hide False Positives: Do users want the ability to hide False Positives from the MR altogether? Would that be at a policy level or from within the MR? How might this affect the inline findings once we've implemented that? (I would assume that the decision taken to hide or show them within the security widget would also apply to the inline findings in the Changes tab).

  • Vulnerability-Check rules: Should Vulnerability-Check block the MR from merging if, say, a Critical vulnerability has been identified as a false positive? The user may want to set this as an option within the policy, e.g. a toggle for "Do not block Critical/ High/ Unknown Vulnerabilities if identified as False Positive. Related to Customize the type of vulnerability in regards to the default branch (e.g., main) for the approval cc @zmartins @sam.white @cam.x

  • Auto-dismissal of False Positives: How might we give users the option to auto-dismiss False Positives? What should be the default behavior here?

  • Likelihood percentages: VET MVC is binary but in the future may have thresholds or likelihood percentages. How and where does this affect the UI? Consider potentially incorporating this into policies as well, e.g. "Auto-dismiss if False Positive is above 95% likelihood"

Proposed changes

  • Add ability for users to include VET FPs in the auto-dismiss rule (opt-in; default to no auto-dismissal)

  • Add ability for users to include False Positives in Vulnerability-Check (opt-out; default to excluding any False Positive that matches the criteria (Critical/ High/ Unknown to default branch) from the rule)

About VET

  1. SAST Analyzer detects vulnerabilities and generates a security report.
  2. VET analyzer works on the generated security report and augments false-positive information in the same report.
  3. Rails backend stores the security report information in the database.
  4. Frontend renders the UI based on GraphQL api call

The false-positive flag identifier is applied just after the security report is generated by the SAST analyzer (for now, only Brakeman).

cc @matt_wilson @tmccaslin @theoretick @farias-gl @ssarka @lkerr @NicoleSchwartz @derekferguson @gitlab-com/gitlab-ux/secure-protect-ux

Edited by Becka Lippert