UX Scorecard: Obtain an overall understanding of job executors (runners) at a glance so that I can make effective decisions
This scorecard is designed to help identify areas of usability and learnability improvements as part of gitlab-com/www-gitlab-com#12084 (closed).
-
Personas
🏃 : Priyanka (Platform Engineer) -
JTBD:
When I am managing the execution of many CI jobs at scale, I want an overall understanding of the job executors connected to my organization, so I can make effective decisions.
- Previous score and scorecard: N/A
- Benchmark score: C (3.00)
- Experience map: Mural
-
Walkthrough video
🎥 : YouTube video -
Walkthrough deck
💻 : Figma slides -
Recommendations
✨ : #1673 (closed)
Evaluation setup
JTBD
When I am managing the execution of many CI jobs at scale, I want an overall understanding of the job executors connected to my organization, so I can make effective decisions.
Scenario
You are responsible for managing GitLab at enterprise scale, ensuring developers in your organization can build, test, and deploy software with ease. To administer a large fleet of runners, you need to have access to insights into the status, availability, security, and performance of your runners in order to make crucial decisions that impact your team of developers, and not block them from their daily workflows. This will include administering runners at both a organization-wide level and group level.
Tasks to complete the scenario:
- View runners associated with an instance or group and verify it is running as desired.
- Search for a runner using its unique identifier.
- Search for an active runner, change it's state to inactive, and then edit it's description.
- Clean up (archive) a large number of runners that haven't contacted GitLab in 30 days.
- Determine how many runners are out of date by 2 versions in order to help with compliance enforcement.
- Obtain a list of all runners used overtime for a specific Project ID
Resources needed to complete
Since we are primarily evaluating the admin view, runners need to be registered on your local version of GDK and you need to follow a similar format to that of https://gitlab.com/steveazz/playground where there are runners with different statuses (paused, inactive).
UX Scorecard Checklist
Learn more about UX Scorecards
-
Add this issue to the stage group epic for the corresponding UX scorecards. Verify that the "UX scorecard" label is applied. -
After working with your PM to identify a top job, write it using the Job to Be Done (JTBD) format: When [situation], I want to [motivation], so I can [expected outcome]. Review with your manager to ensure your JTBD is written at the appropriate level. Remember, a JTBD is not a user story, it should not directly reference a solution and should be tool agnostic. -
Make note of which personas might be performing the job, and link to them from this issue's description. Keeping personas in mind allows us to make the best decisions to address specific problems and pain points. Note: Do not include a persona in your JTBD format, as multiple types of users may complete the same job. -
If your JTBD spans more than one stage group, that’s great! Review your JTBD with a designer from that stage group for accuracy. -
Consider whether you need to include additional scenarios related to onboarding. -
Review the current experience, noting where you expect a user's high and low points to be based on our UX Heuristics. Using an experience map, such as the one found in this template, capture the screens and jot down observations. - If you're re-scoring the experience, review the entire flow, but feel free to reuse existing artifacts (i.e. a UI screen that wasn't changed).
-
Use the Grading Rubric to provide an overall measurement that becomes the Benchmark score for the experience (one grade per JTBD), and add it to this issue's description. Document the score in the UX Scorecard Spreadsheet. -
Once testing is complete, create a walkthrough video that documents what you experienced when completing the job in GitLab. Begin the video with a contextual introduction including: - Your role, stage group
- Specify how you conducted the heuristic evaluation
- Add a short introduction describing the JTBD and the purpose of the UX scorecard (i.e. you're performing the evaluation in partnership with {stage group} and {product designer}.
- This is not a "how-to" video, but instead should help build empathy for users by clearly showing areas of potential frustration and confusion. (You can point out where the experience is positive, too.)
- At the end of the video, make sure to include narration of the Benchmark Score. Examples here and here.
- The walkthrough video shouldn't take you long to create. Don't worry about it being polished or perfect, it's more important to be informative.
-
Post your video to the GitLab Unfiltered YouTube channel, and link to it from this issue's description. -
Link to your video in the Engineering Week in Review. -
Create a recommendation issue for this JTBD and add it to the same stage group epic as this issue. Also add a link to your recommendation issue to this issue.