[Solution Validation]: Usability focused validation for adding/editing job token permissions

What did we learn

Results
Users found the experience of adding and editing job token permissions for a group and project to be usable, straight-forward and easy to use.
Dovetail project

The proposed designs for adding and editing job token permissions for a group and project in the allowlist is "straightforward, clear and easy to use"

Task Pass/fail Average rating for ease (out of 5) User feedback
On this page, configure job token access so that only the Mango project can use your current project's resources. 4/5 4.4
  • "Easy to use"
  • "Straightforward"
  • 1/5 users suggested an improvement the hierarchy of information "moving the save button"
  • 1/5 users who didn't complete the workflow was confused by project being self-allowed in the list
From this page, update the Mango project’s permissions to allow only read and write access to the container. 5/5 5
  • Options are easy and clear

Recommendation: Move forward with the proposed designs with refinement considerations below.

Users expressed familiarity with the token permission options: default vs fine-grained permissions

When asked to describe token permission options, users were able to understand the meaning of both permission options, describing fine-grained as "stricter."

Recommendation: Move forward with UX language in token permissions.

A few other considerations for improvement:

  • 1/5 participant noted a potential improvement would be moving the save button. Design recommendation: Continue to monitor feedback in this area and refine hierarchy as needed
  • 1/5 participant expressed a more simplified copy for the subheading for Job Token Permissions would benefit users who are new to using GitLab. Design recommendation: Explore potential copy improvements.

Background

What’s this issue all about?

From [UX] Design: Adding and editing fine-grained pe... (gitlab#482107), we have an MVC for adding and editing permissions on job tokens.

Who is the target user of the feature?

  • Development Team Lead / Software Developer / Platform Engineer / Security Operations
    • Role in GitLab instance specific to Project Owner / Maintainer

What questions are you trying to answer?

Core questions
  • Is the proposed workflow for job token permissions usable?
  • Is it clear to users where to go to update their job token's permissions?
    • We are assuming that permission definition is implied in the workflow
  • Are the options of read, write and read, none clear for users?
  • What other information is needed to ensure task success?
Usability success measurement
  • Task effectiveness (Pass/fail rate)
    • Observation of why users pass or fail
  • Task efficiency (Ease of rating) 
    • Qualitative explanation of rating
  • Quality of experience (Good or bad)
    • Qualitative explanation of rating

What hypotheses and/or assumptions do you have?

  • Users will find the workflow to define job token permissions to be efficient, effective 

What persona, persona segment, or customer type experiences the problem most acutely?

  • Project owners and maintainer

What business decisions will be made based on this research?

  • Move forward with the MVC proposal for gitlab#482107 (closed) 
  • Generate usability feedback for future iterations and improvements

Past research

  • In this research we learned that gitlab#480756 (closed)
    • API & UI definition for job token permissions with project allowlist is acceptable to participants, provided there's a default set of permissions to ensure existing jobs and pipelines do not break

Test materials

Checklist

  • Draft goals and objectives, review with collaborators
  • Create test plan, review with collaborators
  • Build prototypes
  • Launch study on usertesting.com
  • Analyze and communicate findings
  • Close issue 🎉
Edited by Ilonah Pelaez