Think-aloud test for Security Dashboard changes
What’s this issue all about?
To quickly get feedback to inform our direction and next steps on the proposed layout change to the security dashboard (part 2 baseline).
Research questions
- What are users’ expectations from a security dashboard?
- Do users understand what the different elements on the page are, and are these elements valuable for them? Why?
- Do users prefer our new designs compared with our existing design? Why?
- If so, which of our 2 versions do users prefer, and why?
Methodology
This will be a think aloud study where we ask participants to share their thoughts as they observe the designs (see detailed script below).
As we are considering 2 versions (A and B), we’ll test both with each participant. To minimize bias, we'll have half of our 6 participants see version A first, and half see version B first.
Following going over A and B with participants, we’ll present them with our existing design that’s currently in production (version C). We'll always present our existing version last, to prioritize getting feedback on the new designs, as we are fairly confident that we will want to discard the existing solution.
Layout proposed | A, with line chart | B, with sparklines |
---|---|---|
1 Header, 2 Main, 3 Aside | line chart and header name | sparklines and header name omitted |
Participant profile
4 security analysts plus 4 developers
Script
< Standard test preamble to be added >
Expectations
- We have a page called Security Dashboard. What tasks would you expect to be able to accomplish with the help of that page? Up to 3 tasks.
First impression
[Display version A to first half of participants, then display version B to the rest]
- This page has to do with our security features. Could you please have a look at it for me and describe what you see? [prompt to think out loud, and note: what is the order in which they mention elements on the page? How do they refer to these elements?]
- What would you say is the purpose of this page?
- Who is it for?
- What title would you give this page?
- Would this page be useful or not useful for you? Why?
Vulnerabilities trends area
- Looking at this element [point at the vulnerabilities trends area], can you describe what you’re seeing?
- Is this particularly useful or not useful for you as a _____ ? Why?
[Show the other version]
- This is a slightly different version of this element. Can you describe what you’re seeing here?
- Which version, if any, do you like best? Why?
- Is there a way we could have made this area more useful for you?
Existing dashboard page
[Show our existing page]
- Here is a third version of this page. You may be familiar with it, as it is the one we currently have in production. What are your thoughts on this layout? Please share them out loud.
- Which version do you prefer, this one or the one you prioritized earlier [A or B]? Why?
Wrap up
- Do you have any other feedback for us on this page?
Thank you for you time.