Solution Validation: Refine UX of how custom permissions are displayed
What did we learn?
Results |
---|
While satisfaction metrics is lower for proposed designs, verbal feedback is more positive with the proposed designs noting ease, clarity and logic in the flows. Users expected more information and functionality from the custom role table to help with auditing or editing. |
See more detailed insights below or in dovetail. |
We tested the existing and proposed experience and found:
Satisfaction metrics were lower for proposed designs, verbal feedback is more positive
All users successfully completed the tasks but proposed designs had more positive feedback, descriptions include "good, clear, logical."
Task | Current Designs (score out of 7) | Proposed Designs (score out of 7) |
---|---|---|
Create Role | 6.57 | 6.57 |
Assign Role | 6.8 | 6.7 |
Non admin viewing role | 6 | 5.2 |
Because the majority of tasks in this test were focused for admins, we created a test audience of only administrators. We then asked them to imagine that they are in a non-admin role viewing the member table. Due to this method of testing, there isn't a high degree of confidence in the answers for "non admin viewing role" and more testing is probably needed.
Recommendation: Move forward with the proposed designs
Proposed experience meets baseline information needs
The information available to users when using the custom role feature is sufficient and meets their baseline needs. Users liked how they can scan the member table and see when a role is custom. This helps with checking against their security policies, however..
Users expect more details for auditing
When looking through both the custom role and member table, users requested additional columns for when the role was created, who created the role and when the role was deleted.
Recommendation: Explore the need to surface additional information in the custom role table for when role is created, who created and how to show deleted roles. Understand the needs of admins when auditing role and permissions from the member table.
Users have varying needs for role & permission transparency for their non-admin team members
The proposed version of the design limits the custom role information for non-admin users to just the description. Most users noted that revealing more information may vary between organizations and teams depending on the security policy. A few users wanted more detail to aid in empowering their teams and upholding transparency while a few noted that the information in the designs are sufficient.
Recommendation: The proposed designs meets the criteria of a scalable UI but we need to look into providing custom role definitions and provide organization's an ability to set who views this information.
Users expected an edit functionality
While scanning the custom role table area, most users expected an icon for the ability to edit the custom role. One user brought up a question around who can edit existing custom roles.
Recommendation: Look into the edit functionality and the permission needs around the custom role feature (who should be able to create, edit or delete a custom role?)
What did we validate?
Background and Context
- We have a proposed refinement of how custom permissions are displayed
What are the goals of your research?
- Determine whether the updated designs are an improvement over the current state
- Understand what level of information detail is most valuable to users for custom roles and at what point of the workflow
What hypotheses and/or assumptions do you have?
- The proposal to disclose description of custom roles will be usable and will help users manage and assign the right custom role for group members
- From the member page, users need to only be informed if a role is custom to the system. All other custom role permission detail can remain within settings
What questions are you trying to answer?
- With the proposed refinement, are users able to choose which custom role to assign to their group members
- Is it clear to users where they could find out more details about a custom role's permissions
- Is it important for users to know that a certain custom role is based on a specific standard role (Custom role based on
Guest
, for example) when assigning a custom role? - How do non-admin users leverage the members page to understand and troubleshoot their role and access?
- What other feedback do they have about design and workflows related to custom roles?
What research methodology do you intend to use?
- Unmoderated Usability
What persona, persona segment, or customer type experiences the problem most acutely?
- Sidney, Systems Administrator
- Priyanka, Platform Engineer (Maintainer)
- Sasha, Software Developer (Non Admin User)
What business decisions will be made based on this information?
- This will help inform and improve usability issues that may prevent adoption
What, if any, relevant prior research already exists?
- N/A
Who will be leading the research?
- @ipelaez1 and @moliver28
What timescales do you have in mind for the research?
- Midway through %16.8
Relevant links (problem validation issue, design issue, script, prototype, notes, etc.)
- Design Issue
- Testing Plan [completed]
- Usertesting tests:
- [B] Proposed
- [A] Current Experience
Progress Checklist
-
Create testing plan -
Review testing plan with team (UX/UXR + Product) -
@moliver28 (UXR) -
@jrandazzo / @hsutor (Product)
-
-
Finalize testing plan -
Build out any needed designs/prototypes for testing -
Create usertesting test -
Synthesize findings & share out results