UX audit: Vulnerability Report's activity items

Problem to solve

UX management conducted a heuristic evaluation of VE and VR (Threat Insights' AI features), and one problem identified in AI Heuristic Evaluation > Vulnerability resolut... (#462207 - closed) was:

When viewing vulnerabilities, there is no indication on the table view that you can resolve the vulnerabilities.

In https://gitlab.com/gitlab-org/gitlab/-/issues/407275+ we discussed adding badges as indicators that AI is available on particular vulnerabilities as a potential solution to this. However, this uncovered a much-needed step back to look at the purpose of filtering by an activity item, and an audit of the colors (or variants) that the badge component offers.

What are the primary purposes of the activity column?

1. Get a "sneak peek" of activity taken (either by the system or by the user) on each vulnerability from the Vulnerability Report view.
   1. This allows me to see if an issue or MR has already been opened by myself or another member of the security team. This saves me time and frustration from having to click into each vulnerability and wait for loading time, in order to view this information.
   2. The context of activity items can also help me with the triage process. For example, I may want to prioritize vulnerabilities that don't have an MR already open proposing a fix, because I can assume those are being handled already by another team member.
2. Filter by the positive or negative of each activity item.
   1. For example, it let's me only see vulnerabilities that are still detected, or only have Vulnerability Resolution available. Let's say that I just implemented a Vulnerability Management policy to auto-dismiss vulnerabilities that match certain criteria; this filter allows me to confirm that those vulnerabilities were dismissed or resolved properly and that there aren't any errors (that the policy is working as expected).
   2. When we start archiving vulnerabilities as part of our upcoming retention policy, we need to let users view which vulnerabilities will soon be archived so they have a chance to triage them before they're only available through the archive export.

Current	Former proposal	New proposal

Part I of II: Define the goals of activity badges

JTBDs

• When I'm using the Vulnerability Report, I want to be able to filter by a particular action or status of that vulnerability, so I can look at vulnerabilities relevant to my workflow.
• When I'm using the Vulnerability Report, I want to be able to HIDE [filter out] irrelevant vulnerabilities, to focus only on the vulnerabilities relevant to my current task.
• When I'm filtering on the Vulnerability Report, I want the UI to confirm that my filter has been applied, so I know the tool is working properly and to my expectations.

As I see it, there are 3 categories of activity items:

Manual actions taken on the vulnerability	System or automated actions or status updates	Feature offered on the vulnerability
What kind of action has someone on my team taken related to this vulnerability? Examples: Has issue [someone opened a related issue] Has MR [someone opened a related MR]	What has changed about this vulnerability on the backend, without any user input? Examples: No longer detected [e.g. "at one point, a vulnerability was detected in a pipeline, but since then, a subsequent pipeline ran and did not find this vulnerability again"] A designer in our team design review this morning suggested that the action here is actually that the vuln has been removed, NOT that the vuln is `still detected` (the current default filter). Maybe we should only show a filter if users are looking specifically for vulns that are "No longer detected" but we don't need to offer the inverse. Solution available [the scanner has offered a suggested solution]* Reopened ["at one point, a vulnerability was detected in a pipeline, it was moved from Needs triage to a closed state (Dismissed or Resolved), but it's re-appeared"] [not yet implemented; see related design issue] False positive [the scanner has identified this as a false positive; you can most likely dismiss it because it's low risk.] [I think this was implemented, but can't be sure if it's still around because I can't find it in any projects. Design issue here] Note: This activity item may also be confused with the dismissal reason of "false positive", which has nothing to do with this badge. I tried to explain this in the tooltip copy in the activity filter here. Auto-resolved [a policy automatically resolved this vuln] [not yet implemented; design issue here] Auto-dismissed [a policy automatically dismissed this vuln] [not yet implemented; design issue here] *This could now be confused with our Vulnerability Resolution offering, where AI offers a solution. How would users know the difference between a solution proposed by the scanner and a solution proposed by the AI? (Keeping in mind they could be the same, or they could be different, solutions).	I turned on AI, now what can it do to help me with vulnerability management here? Example: GitLab Duo AI help is available (VR/ VE)

Part II of II: Come up with badge variant logic

From Pajamas:

• Neutral muted (default): Metadata that requires the least amount of visual emphasis and has a neutral meaning.
• Neutral: Metadata that has a neutral meaning.
• Info: Metadata that's informative or new and also has a neutral meaning.
• Success: Metadata that communicates success or completion with a positive meaning.
• Warning: Metadata that requires attention and a slightly negative meaning.
• Danger: Metadata that indicates a problem and has a negative meaning.
• Tier: Metadata that indicates which product tier an entity belongs to *

*We can ignore this one as we don't specify tier in the product UI; it either shows up if they meet the qualifications (e.g. AI is enabled or the feature is scoped to an Ultimate license) or it doesn't.

Let's contextualize these with some examples in the product:

Neutral muted	Neutral	Info	Success	Warning	Danger	None?
1.Badge count in tabs - Vulnerability Report shown here 2.Merge requests page 3.Kubernetes clusters 4.Analytics dashboard 5.Releases	1.Deploy > Model registry 2.Security configuration 3.Branches 4.Value stream analytics 5.Jobs page and	1.Jobs page 2.Preferences 3.Merge requests page 4.Branches 5.Kubernetes clusters	1.User settings > emails 2.Jobs page 3.Merge requests	1.Dependency List 2.Kubernetes clusters	1.Requirements 2.Merge requests page 3.Analytics dashboard	Profile > contributed projects

Let's try to correlate each proposed and current activity type to these variants:

Activity item	Notes	Badge
Detection	I propose that we move away from this being a filter at all after the following 2 features are implemented: After we have the policy to auto-resolve vulns that are no longer detected. With this policy, users won't need to interact with a still detected/ no longer detected filter and won't be a day-to-day decision. The majority of our users are going to implement this policy [an educated guess but not validated] because we hear that the number of detected vulns can be overwhelming and they're always looking for ways to reduce the "noise". As part of adding CVSS, KEV, and EPSS, I'm proposing a drawer to customize the Vulnerability Report view; I'd like to add a toggle in this drawer where users can choose to include no longer detected vulns. (It would default to off). If the user toggles this ON, we have some options: 2a: Don't show any confirmation in the UI. This is dangerous because it's essentially a hidden filter that has been applied with no confirmation to the user that they're looking at vulns that are still detected or no longer detected. 2b: Show the "no longer detected" icon in the activity column. This would solve the following problem: We're already not following the logic of the Activity column (if the first item in each category of the activity item has a filter, confirm it with an icon in the activity column), because we're defaulting to showing only Still detected now, and yet we're not showing a related badge in the Activity column. Rather than show a "still detected" icon, we show the inverse (where it's no longer detected). The decision to show "no longer detected" vulns would be a manual decision made by the user (per the toggle). In this case, I don't think we need any changes to the current "no longer detected" icon seen to the right. It shouldn't be the success variant because the user has to confirm that the vuln was, in fact, removed or resolved. It's supplemental information that the user should be aware of during triage, so the existing blue works. 3. Keep Detection in the Activity filter and don't include it as part of the drawer (with KEV, etc). My proposal moving forward is: Any activity the system took should be blue Any activity the user took should be neutral Because the system no longer detects the vulnerability, this remains blue as long as we have a badge denoting that a vulnerability is no longer detected.
Issue	Neutral muted - Because the user added a related issue to this vulnerability, this remains neutral muted (the default).
MR	Neutral muted - Because the user added a related issue to this vulnerability, this remains neutral muted (the default).
Solution available	Info - Because the system/ tool is suggesting the solution and thereby took the action of adding a solution, this should be updated to blue.
False positive	remove (design never reached production and has since been deprecated)
Reopened	remove (design never reached production and has since been backlogged if not deprecated)
Auto-resolved	Success - The vulnerability was no longer detected or matched other criteria (it was in a file or matched an identifier) that the user told the system to do. We can thereby assume that the system completed the action (following the policy in order to better manage vulnerabilities at scale) with a positive meaning.
Auto-dismissed	This could go 1 of 3 ways: It was a system action, so should be blue. However, it was also (kind of) a: manual action that the user took (creating the auto-dismiss policy and setting the criteria), so it should be neutral muted. However, if we want to differentiate this from the other neutral muted badges, we could use the neutral variant. manual action that the user took (creating the auto-dismiss policy and setting the criteria), but different from the other types of direct, manual action a user can take (relating an issue or MR), so we should use the other neutral variant.	or or
Soon to be archived	(Future activity item - see Design: Data retention UX (#464638 - closed)) Warning - Requires attention and has a slightly negative meaning.
AI	Info - Because the system/ tool is offering AI assistance, this should remain blue.

Proposal

Success (green) and warning (orange), are fairly self-explanatory on guidelines for use. Therefore, I think we need to come up with a rule for the other neutral statuses, the two neutral variants (muted, neutral) and info. My proposal is:

• Blue = Any activity the system/ tool took, or a system/ tool-generated feature that is made available to the user.
  • This is important to emphasize with a non-neutral color because the system has taken action or is offering action for the user to take, such as a solution available or AI available, or a subsequent pipeline no longer found the vulnerability.
• Neutral muted = Any activity the user took.
  • This is the least important to emphasize because it's confirmation of an action that the user took, such as opening a related issue or MR.
  • After meeting with @jmandell on 2024-11-12, we agree that there's not enough visual differentiation between the neutral and neutral muted badges to use each, so all activity-created activity items will use neutral muted.

Actions to take:

• Open an issue to change the Solution available badge from neutral muted to blue. Captured in: #504502
• The proposal for the auto-dismiss activity badge changes from neutral to neutral-muted. Captured in: #299552[design_1731626897647.png]

The new activity item badges (auto-dismiss, auto-resolve, and soon-to-be-archived) should take place in the corresponding issues.