UX Scorecard - Optimize FY25-Q1 - Create a quarterly progress report / Report on progress
Summary
Grade | |
---|---|
Average |
- Personas/Job Performer: Delaney (Development Team Lead) / Progress Reporter
- Main Job: Track plan progress to meet commitments
-
Small Jobs:
- Put together a report
- Synthesize information
- Summarize information
- Export / Import data to another system (as a work-around, this could be in the form of screenshots or copy & paste)
- Job Story: When I need to communicate my team's performance to leadership, I want to create a report, so I can increase alignment toward achieving goals.
- Scenario: You have been tasked to create a quarterly progress report for your team's performance. Using Value Stream Analytics, create a report tracking the amount of deployments your team completed in the last quarter along with the average time it took the team from opening a merge request to merging the merge request.
- Link to FigJam canvas
- Previous score and scorecard: n/a
- Benchmark score: n/a
- Scorecard approach: formative evaluation
- Walkthrough video: https://youtu.be/inv-Z8ddAHU
- DoveTail project: https://gitlab.dovetail.com/projects/CmY7FmE5eABarHZC4k359/readme
- Recommendations: Experience Recommendations - Optimize FY25-Q1 -... (gitlab-design#2556)
Findings related to the scenario
- All 3 of the participants were able to complete the tasks with some struggling.
-
“I thought this would be easier than it was”
-
- When looking for the metrics in the UI all participants seemed to struggle to understand what the Value stream dropdown option did and that the path bar was clickable.
-
“One thing I always that threw me off is I know like tabs that show you like the the life cycle. It's not super obvious like from a visual cue perspective.”
-
- When asked what they would do next, after finding the metrics in the UI, all participants expected to see some sort of export or embed option on the screen.
- When asked a follow up question on whether the number of deploys metric was relevant to their workflow and they mention that it would be more useful to see the average number of deployments over time vs a total count.
Additional findings
- 1/3 of the participants used VSA before
- All participants were aware of the term DORA metrics but no one used them
- All participants used Sisense dashboards and were now using Tableau or were in the process of transitioning to Tableau
- Some teams work is split across multiple groups making it a manual task to sum up the number of merge requests, merged at the end of a milestone
- None of the participants actually create reports
- Metrics participants were interested in:
- MR reviews
- How long MRs are open
- Number of issues closed
- How many issues were assigned vs completed in a milestone
- Vulnerability SLOs
UX Scorecard Checklist
Expand
Learn more about UX Scorecards
-
Add this issue to the stage group epic for the corresponding UX scorecards. Verify that the "UX scorecard" label is applied. -
Work with your PM to identify a top Job to be Done (JTBD). All GitLab JTBD can be found in the jobs-to-be-done.yml file. If creating a new job, write it using the JTBD format: When [situation], I want to [job], so I can [expected outcome]. Review with your manager to ensure your JTBD is written at the appropriate level. Remember, a JTBD is not a user story, it should not directly reference a solution and should be tool agnostic. -
Make note of which personas might be performing the job, and link to them from this issue's description. Keeping personas in mind allows us to make the best decisions to address specific problems and pain points. Note: Do not include a persona in your JTBD format, as multiple types of users may complete the same job. -
If your JTBD spans more than one stage group, that’s great! Review your JTBD with a designer from that stage group for accuracy. -
Add any new JTBD to the SSOT jobs-to-be-done.yml file -
Consider whether you need to include additional scenarios related to onboarding. -
Select the appropriate scorecard approach and evaluate the current experience. -
Use the Grading Rubric to provide an overall measurement that becomes the Benchmark score for the experience (one grade per JTBD), and add it to this issue's description. Document the score in the UX Scorecard Spreadsheet. -
Once testing is complete, create a walkthrough video that documents what you experienced/witnessed within the existing experience. Begin the video with a contextual introduction including: your role, stage group, specify how you acquired the data (ex: internal or external users, or self-heuristic evaluation), and a short introduction to your JTBD and purpose of the UX scorecard. This is not a "how to" video, but instead should help build empathy for users by clearly showing areas of potential frustration and confusion. (You can point out where the experience is positive, too.) At the end of the video, make sure to include narration of the Benchmark Score. Examples here and here. - If you're re-scoring the experience, walkthrough the entire flow again. For narration, you can highlight the recent improvements but still call out any areas that could still use some tweaking (in the next round of iterations, if applicable). The re-score video, in theory, should be shorter since we've hopefully eliminated a few bumps in the user flow.
- The walkthrough video shouldn't take you long to create. Don't worry about it being polished or perfect, it's more important to be informative.
-
Tag PM and UX DRIs for this JTBD in this issue to share findings. -
Post your video to the GitLab Unfiltered YouTube channel, and link to it from this issue's description. -
Link to your video in the Engineering Week in Review. -
Create a new Dovetail project using the UX scorecard template. Use insights to document any observations or findings that came out of this scorecard. You can use your experience map or video summary to help you curate those. It is important to add insights into Dovetail so they can be shared and accessed by all groups, and used to document cross-stage findings. You can also add any supporting material in Data, such as an exported Mural experience map, but it is not required. Example here. -
Create a recommendation issue for this JTBD and add it to the same stage group epic as this issue. Also add a link to your recommendation issue to this issue. - [ ]
Edited by Libor Vanc