Research: Secure: MVC Secret detection

  • Researcher: Camellia
  • Milestone: 12.9
  • Note: It will be in one research session with this issue: #680 (closed)

Business Decision:

  • What is the best way to help the user identify Secrets-vulnerability
  • What is the best way to help the user triage Secrets-vulnerability

Hypothesis:

  1. MVC-standing alone Secrets Page will help users:
    • User needs a separate page because Secret is very different
    • User needs special actions to triage Secrets: view commit/check details in commit page

  1. MVC-combined: Secrets integrated in "Dashboard":
    • This is good enough for the user, secrets don't special treatment compared to other vulnerabilities

Hypothesis metrics:

  • Did the user think Secret is more important than other vulnerabilities
  • Did the user triage Secret different? how?
  • Did the user need "commit" or other information on list page for easier triage?
  • Did the user need "add to whitelist" feature?

Goals:

  • Find out what is the best way to display Secrets
  • What information does the user need to triage Secrets
  • Is the current design is good enough for MVC: dashboard or standing alone secret page

Objectives:

We want to get the following major question answered:

  • How "Critical" Secrets are?
  • What actions user need to take to triage "Secrets"?

Participant profile:

Security Analysts (Gitlab Persona: Sam (Security Analyst)) who have experiences manage Secrets

Research Script

https://docs.google.com/document/d/1z-wlUJ3EqpA5Oq-3dQQmJqre61Z6o6Y502SEUhyJ5N4/edit?usp=sharing

Rainbow Note

https://docs.google.com/spreadsheets/d/1ej37tJkCKUZSpG7EZQLK-LyzpXLTOESui6KHs--A14s/edit?usp=sharing

Recruiting issue:

https://gitlab.com/gitlab-org/ux-research/issues/707

Screener:

https://docs.google.com/document/d/1-Zkg2mkivogH2rEHzvYna5GTGNFnFp2GICzgFVIeaRI/edit?usp=sharing

Research result

Highlights
  • For those who have no experiences before triaging Secrets, but has Security background (2 people): Use the information we provide to make a decision:
    • When Secrets display on the dashboard with other vulnerabilities and severity label: critical. They think Secrets are important to deal with.
    • When Secrets display on standing alone page, they think they are not vulnerabilities, it is a place to manage Secrets
  • For those who have experiences triaging Secrets (2 people): have a different option about Secrets:
    • One think Secrets is definitely vulnerabilities and should be triaged first. But prefer it to be in the dashboard so that she can manage everything together
    • Another one thinks not all Secrets are vulnerabilities, because they have different ways to protect exposed secrets. So prefer the standing alone list to manage Secrets.x
  • No one really think commit is the most important info to triage Secrets
  • Whitelist feature is somehow confusing

Raw interview materials: link

UXR insight project: Epic: link

Design decisions and extra ideas

  • For MVC design, we keep secrets in the Dashboard list. Add scanner column to help the user identify this is Secrets.
  • For future design, we need to understand the needs of users:
    • For those who have no experiences before triaging Secrets: Do user needs education about triage secrets
    • For those who have experiences triaging Secrets: Do think Secrets as the same we define it? Is "Secrets Manage" is a validate option

Extra ideas:

  • Sorting by time
  • Add assignees in the list view
  • The intro about Secrets(educational)
  • Auto remediation for Secrets -> one button kill the secrets
  • Different naming “Secret keys”
  • Secrets management
  • Whitelist is more general manage features, whitelisting has different definitions in the industry

Research task-list

  • (Designer): Create research issue
  • (Designer): Sync plan with PM/Researchers
  • (Designer): Create recruiting issue
  • (Designer): Create Screener
  • (Designer): Sync with Research coordinators
  • (Designer): Create Calenderly link and share with Research coordinators
  • (Coordinator) Start recruiting
  • (Designer): Create Script
  • (Researcher) Review Script
  • (Designer): Create rainbow note taking sheet
  • (Designer): finalize prototyping
  • (Designer): make sure this is a note taker
  • (Designer): dry run/pre-research sync meeting
  • (Designer): Conduct one usability testing session. Amend script if necessary.
  • (Designer): Conduct remaining usability testing sessions.
  • (Designer): Open an Incentives request. Assign it to the relevant Research Coordinator.
  • (Coordinator): Pay users.
  • (Designer): Synthesize the data and identify trends, resulting in findings.
  • (Researcher): Review findings and provide feedback, if needed.
  • (Designer) : Create issues in the UXR_Insights project documenting the findings.
  • (Researcher): Sense check the documented findings.
  • (Researcher): Update the Solution validation research issue. Link to findings in the UXR_Insights project. Unmark as confidential if applicable. Close issue.
Edited by Camellia X Yang