Today we 2 security dashboards and are planning to add a third one soon (and a fourth!). For the most part, the dashboards present the same information to the user with slight variations based on where the dashboard is located (project, group, instance). We also don't define the use and use cases for each dashboard, relegating them to a catch-all for project security features and workflows. This current approach is slowly leading to overloading of our dashboards and could be confusing users as to the tasks they need to complete from these views.
Problem:
Lack of prescribed workflow has led us to design dashboards that act as a catch-all for our security features.
How might we leverage one security dashboard for users to use while providing the appropriate information to users in specific areas of the application?
Lack of dashboard framework makes it difficult for multiple designers to contribute to the experience.
How might we create and maintain a dashboard framework so that contributing to the experience is easier and doesn't cause regressions or rework?
Process:
Collaborate on a comprehensive dashboard strategy
Create: Journey maps, flows, JTBD, and diagrams as needed
Refine: Strategy and artifacts used
Document: Strategy in Secure UX Handbook
Communicate: Strategy to Stage and Org.
Collaborate on dashboard framework
Work with @sbouchard1 on the unified dashboard initiative
Create: Concepts for the security dashboard based on collaboration direction
Refine: Concepts and mocks as required
Document: Framework rules and guidelines as required
I think the first place to start here is to nail down the various personas we have for Secure (and Defend) features and the specific JBTD they have to guide us to improving the dashboards. It's very possible we'll end up with more than one, depending on the persona.
Some initial thoughts on who to will be using the dashboards:
App developer
Builds features and fixes bugs for a specific project or app in their organization.
Wants to know what is outstanding with the related project to understand what impact they are having or what vulnerabilities/security issues need to be addressed
May work at a small, medium, or large company
App portfolio manager
Responsible for managing multiple apps and projects, coordinating delivery of each, and making sure company security & compliance policies are followed
Very short on time since has many projects to manage
Wants to understand which projects are in the most critical state to help focus efforts
Wants to understand patterns over time to assess organization health and trends
Operations engineer
Responsible for ensuring a given app is consistently available, minimize downtime, and be aware of any security issues that need immediate attention
Wants to understand what sorts of security events are occurring so that they can take action if needed, either proactively or reactively and prioritize anything that needs to be done during maintenance windows
End-users for SDK projects (this one needs some more fleshing out I think)
Wants to integrate the SDK into their own app or SDK
Wants to understand the security posture of the SDK to understand what they will be exposing their own app to
Wants to make a decision if they are ok with using the SDK or not based on the results
Solid @stkerr Thank you! Starting at the user level helps a lot. It seems like there is a nice divide of the types of tasks we need to support. From what I've gathered, there are essentially two types of dashboards we need to consider, task-based and overview based.
Attributes
Dashboard type
Interaction type
Data
Insights
Timeframe
Goal
Task-based
Deep, multi-faceted
Granular
Actionable
What's happening now
To take action based on information
Overview-based
Limited
Broad
Directional
What happened in the past
To tell a story based on data
! These are not absolutes and can vary based on use case
Task-based dashboard
Designed to facilitate immediate action and informed decision making. A security engineer will use this dashboard to triage, resolve and/or assign active security findings for remediation.
Typically list centric or with list components. Can have metrics, only if they help the user with their task. Users may spend a good portion of time here.
Designed to provide directional data and insights. A director will use this dashboard to report on the company's security performance for the quarter. An auditor might use this dashboard to view historical events and resolutions.
Typically chart or data-centric. Always has metrics and may have a list component though the list should support the user complete their goal. Users may spend only a few minutes here.
Personas who would use this dashboard:
App portfolio manager, Sec engineering manager, Security Director
@andyvolpe I really like the separation here! It seems spot on.
What happened in the past
For this part of overview-based dashboards, I don't think this is always true. There are use cases where an individual will look at the current-state dashboard and use that to know how/when to drill into a task-based dashboard. For example, an app engineer could the live state of 20 apps, identify 1 risky one, then dig into the relevant task-based dashboard from there.
An auditor might use this dashboard to view historical events and resolutions.
This is a good point - auditors are a whole persona we will need to build out, since audits are a big part of the security lifecycle.
@andyvolpe I'm wondering if we really need two different dashboards for this, assuming we're talking about dashboards as a page that houses the content. For example, say there's a graph (widget) showing a historical overview of the number of vulnerabilities found over the past week, and below it (in another widget) the list of vulnerabilities to review. Aren't these different widgets but both on the same dashboard?
I tend to agree with @beckalippert, but I could be missing something and lo-fi wires might help me here...
Maybe it's the terminology I'm getting hung up on - task vs. overview, action vs. tell a story. To me they're all actionable dashboards - whether the action is triaging security issues or sharing dev metrics to fuel continual improvement. That said, I agree the personas, action type(s) and frequency of use should influence the solution.
One grid but maybe several dashboard templates for dashboard types? - I'm assuming one dashboard grid can be used to create actionable dashboards of all kinds. It's just a matter of which responsive chart and or table widgets are added, and and how they're laid out. That said, I could see dashboard templates applied to certain types of dashboards.
I'm wondering if we really need two different dashboards for this
Bigger discussion than what is outlined in this issue:
I agree we need to determine if multiple dashboards are needed. Have we considered allowing custom dashboards with a library of widgets that each user can set? At that point, it's whatever they want it to be based on what we offer. The what we offer part is what we could then focus on to make sure we provide the right data and information, in a useful way (based on the personas mentioned above).
I don't know if Secure has discussed custom dashboards, but Analytics has discussed default vs. custom dashboards. I think Monitor may already support some sort of customization.
A GitLab Report Widget Library would 1) enable GitLab teams to release more consistent dashboards, faster and, 2) give us the option to expose that same library to customers at some point so they could customize default dashboards and or create dashboards from scratch. The UX for customizing dashboards might look something like this:
I start with a dashboard, and can resize, drag and drop, and add and remove widgets to customize:
I can also go to the GitLab Report Widget Library to pull in new widgets:
If we can build report widgets and default dashboards that solve 80% of common needs, then we deliver immediate value for all and set ourselves up to deliver customized dashboards too.
Custom dashboards are definitely the way to go! Not only is it flexible but it could help us with our current debt of three dashboards, one for each level, project, group, instance. As you can imagine, it's a difficult position to be in. We have an old issue still open to explore this: #9315
To clarify an earlier point about 2 dashboards:
Maybe it's the terminology I'm getting hung up on - task vs. overview, action vs. tell a story. To me they're all actionable dashboards - whether the action is triaging security issues or sharing dev metrics to fuel continual improvement. That said, I agree the personas, action type(s) and frequency of use should influence the solution.
They are indeed all actionable at some level through the depth of the experience may vary. Perhaps we need a better definition of Dashboard. In my mind the security dashboard we have today at the project level isn't a dashboard at all, it's a list. There's no data aggregation, health/performance metrics, overview information, security KPIs, or historical metrics. Just a list of vulnerabilities and counts of vulns by severity type. Maybe we want to create a definition that combines both task and overview dashboards? We should consider what the attributes of a dashboard are and use this to base a definition off of.
@andyvolpe - That context helps. Thanks! Sounds like we agree:
We should start with the users and their goals, not the data
We may want to define what a dashboard is and isn't
We may want to define dashboard types based on TBD attributes
Wondering if 2 and 3 are best solved as we design and discuss dashboards as a UX team. I tend to think of dashboards as pages with visualizations, but perhaps we decide dashboards are pages that use the dashboard framework and 1 or more reusable report widgets - even if that report widget is just a table.
Note: There's plenty of blogs out there defining dashboard types like Operational, Analytical, Strategic, Platform, etc, which may or may not be useful to us.
Cross referencing our interviews study for Security Operations, as it is likely to produce insights which would contribute to shaping our JTBDs for the task-based dashboard.
Side-note: suggesting we refer to the budding persona of sec ops as just that - Security Operations Engineer, rather than Operations Engineer, to differentiate security-oriented operations professionals from SREs. @stkerr WDYT?
@tlavi it sounds like a good distinction to me. I suspect in some organizations, we'll see people who may be part SecOps engineer and part SRE & in others we'll see people who are 100% SecOps engineers or 100% SREs.
@stkerr@vkarnes Thinking more about this, this issue should be considering the Defend area in the conversation as well. I am going to push this out until we are ready to regroup and defend features are established.
FWIW the conversation is progressing nicely, though I believe we won't have the full picture until we get at least the WAF and RASP to minimal/viable.
@andyvolpe Wondering if I can use this issue to document the discussion that's taken place regarding the IA of Security & Compliance in the nav. Per my 10/23/2019 Secure UX Agenda item that was briefly discussed and noted as needs followup:
We are currently on track to having dedicated pages for Vulnerability Management, Threat Monitoring, and the Dependency List. Do we maintain this hierarchy to let the user dive in (A - see below) or do we consider nesting them under the Security Dashboard and supporting deep dives (B - see below)?
A
I. Security Dashboard
II. Vulnerability Management
III. Threat Monitoring
IV. Dependency List
V. License Compliance
VI. Configuration
B
I. Security Dashboard (includes Vulnerability Management, Threat Monitoring, and the Dependency List)