2019 Q2 - Secure UX scorecard: Interacting with vulnerabilities in the MR

Interacting with vulnerabilities in the MR

JTBD: When committing changes to my project, I want to be made aware if I am adding risk through vulnerable code, so that I know my changes can be merged without increasing the risk of my project.

Checklist

• 1. Document the current experience of the JTBD, as if you are the user. Capture the screens and jot down observations. Also, apply the following Emotional Grading Scale to document how a user likely feels at each step of the workflow. Add this documentation to the epic's description.
• 2. Use the Grading Rubric below to provide an overall measurement that becomes the Benchmark Score for the experience, and add it to the epic's description.
• 3. Once you’re clear about the user’s path, create a clickthrough video that walks through the experience and includes narration of the Emotional Grading Scale and Benchmark Score.
• 4. Post your video to the GitLab Unfiltered YouTube channel, and link to it from the epic's description.
• 5. If your JTBD spans more than one stage group, that’s great! Review your JTBD with a designer from that stage group for accuracy.
• 6. Create an issue to revisit the same JTBD the following quarter to see if we have made improvements. We will use the grades to monitor progress toward improving the overall quality of our user experience.

---

Scoring guidelines:

1. Document the current experience of the JTBD, as if you are the user. Capture the screens and jot down observations. Also, apply the following Emotional Grading Scale to document how a user likely feels at each step of the workflow. Add this documentation to each JTBD issue's description.

• Positive: The user’s experience included a pleasant surprise—something they were not expecting to see. The user enjoyed the experience on the screen and could complete the task, effortlessly moving forward without having to stop and reassess their workflow. Emotion(s): Happy, Motivated, Possibly Surprised
• Neutral: The user’s expectations were met. Each action provided the basic expected response from the UI, so that the user could complete the task and move forward. Emotion(s): Indifferent
• Negative: The user did not receive the results they were expecting. There may be bugs, roadblocks, or confusion about what to click on that prevents the user from completing the task. Maybe they even needed to find an alternative method to achieve their goal. Emotion(s): Angry, Frustrated, Confused, Annoyed

2. Use the Grading Rubric below to provide an overall measurement that becomes the Benchmark Score for the experience (one grade per JTBD), and add it to each JTBD issue's description.

Grade	A (High Quality/Exceeds)	B (Meets Expectations)	C (Average)	D (Presentable)	F (Poor)
Description	Workflow is smooth and painless. Clear path to reach goal. Creates “Wow” moments due to the process being so easy. User would not hesitate to go through the process again.	Workflow meets expectations but does not exceed user needs. User is able to reach the goal and complete the task. Less likely to abandon.	Workflow needs improvement, but user can still finish completing the task. It usually takes longer to complete the task than it should. User may abandon the process or try again later.	Workflow has clear issues and should have not gone into production without more thought and testing. User may or may not be able to complete the task. High risk of abandonment.	Workflow leaves user confused and with no direction of where to go next. Can sometimes cause the user to go around in circles or reach a dead end. Very high risk of abandonment, and user will most likely seek other methods to complete the task.
Frustration	Minimal to none	Low	Medium	High	Very high
Task completion	Successful	Successful	Successful but with unnecessary steps	Unlikely, but there may be a chance that there is completion	very unlikely
Steps to accomplish task	Minimal	Minimal	Average complexity	Excessive	Lacking

---

Experience:

Overall Experience Score: D (Presentable)

Workflow has clear issues and should have not gone into production without more thought and testing. User may or may not be able to complete the task. High risk of abandonment.

📹 Link to walkthrough

Making a change and going to the MR.

Sub-Grade: Neutral

The user’s expectations were met. Each action provided the basic expected response from the UI, so that the user could complete the task and move forward. Emotion(s): Indifferent

Let's start by making a change to my project in a feature_branch and running a pipeline with security jobs.

Pipeline is running with security jobs
✅ So far so good. My pipeline is running and I see the results of the security test.

Great, tests are passing and I am starting to see results. Let's move on 👇

Understanding and assessing security risk

Emotional grade: Negative

The user did not receive the results they were expecting. There may be bugs, roadblocks, or confusion about what to click on that prevents the user from completing the task. Maybe they even needed to find an alternative method to achieve their goal. Emotion(s): Angry, Frustrated, Confused, Annoyed

Now that the pipeline has finished running, I can dive into the results and see what is what.

Security results in MR
I see that the Security scanning has detected vulnerabilities in my branch.
❗ Are these vulnerabilities all from this recent commit? They can't be, can they?
❗ I see that these are for the source branch only so I guess they are mine. I wish that was more clear.

Expanding the list to see the results
❗ This list is hard to process, what is my main priority here?
💡 Maybe going to the Full Report will help me with this list.

Goes to Full Report for a better view of the security results
❗ Wait, this is the same information and the same layout as the report in my MR. Why is this helpful?
🔙 Back to the MR where I can work to get this merged in!

Back to the security results list in MR
❗ I am not sure where to start… I guess with SAST since it’s first.
❗ I see a lot of X’s what do those icons mean? They are all the same.
❗ I see the labels Medium (High): what does that indicate?
❗ There is a link with a description a link for the location… which one do I start with.
❗ Clicking the file link takes me away from the page? why is it so prominent? I want to stay here and resolve these problems.
❗ Oh there are more if I scroll in the list area. I didn’t know that.
❗ Why are some vulnerabilities crossed out?

Interacting with vulnerabilities

Emotional Grade: Negative

The user did not receive the results they were expecting. There may be bugs, roadblocks, or confusion about what to click on that prevents the user from completing the task. Maybe they even needed to find an alternative method to achieve their goal. Emotion(s): Angry, Frustrated, Confused, Annoyed

Let's dive into these vulnerabilities and see what we can do:

Clicks on top vulnerability in SAST
❗ These are in modals? Do I have to open these one at a time? this might take a long time!
❗ This one already has an issue created for it. I guess I can ignore that one. It would be nice to know that before I clicked on it.
❗ What does class mean? How does that help me?
❓ Are identifiers more information about the vulnerability? Let's find out

Clicks on an identifier in the modal
❗ Oh, clicking on an identifier took me away from the site. All other links don’t do that 😞 At least it opened in a new tab.
✅ Okay, so I know how to fix it. Let's go back and see if I can fix it.

I guess I'll go into the file and address the problem

🔙 to MR - Clicks on the file that is affected.
❗ So what am I supposed to do next? Fix it here? The file link takes me away from the MR
❗ I can't fix the problem because I am not on a branch?

Okay, I don't have a lot of time right now, let's create an issue and move on.

Creates issue from vulnerability
❗ Oh, it took me away from the MR, not what I expected. I need to fill out this issue with labels and all the other information required by my organization, but I need to get back to that MR!
🔙 There's no simple way to go back! I'll use the back button I guess.
❗ Back where I started? I've lost my place.

I am not sure I can do this for 65 vulnerabilities.

---

Recap:

Overall Experience	Making a change and running a pipeline	Assessing security results	Interacting with vulnerabilities
D (Presentable)	⚫ Neutral	🔴 Poor	🔴 Poor

What didn't go well:

There are a few categories that need to be addressed here for the baseline score to improve. In a follow-up issue I will detail proposed changes to the experience that address issues found here.

Feedback and communication design	Layout	Interaction Design	Workflow
Icons are not clear	List is in an odd structure making it difficult to parse	Users should not be able to create issues from dismissed vulnerabilities	No clear path we want users to take when attempting to fix a vulnerability.
Severity and confidence labels are presented in a confusing manner	Multiple scrollable sub-lists make the experience feel cumbersome	Modals are not optimal for this experience	Users lose their place in the security section if they click on the file path and then return to the MR
Dismissed vulnerabilities are presented in a confusing manner		Links to files take you out of the branch and into an area where they cannot be edited / fixed.
Vulns with issues are not indicated in list
Unclear labels (class) provide information to users that might be confusing
We should bring in as much info from CWE and other similar sources to reduce the need of having to leave the application for more details about the vuln.