UX Scorecard - Release:Release Management - FY20-Q4 - Create a Release and update it (revisited)
This issue is designed to track progress and updates to the MR following the Baseline Evaluation & Recommendations.
- Personas: Persona: Release Manager, Persona: DevOps Engineer involved in executing releases, Automation engineers who are working on releases, Users of a project who aren’t necessarily part of the core team - This can be people who use a project's code/binary but are not otherwise involved, and just want to be aware when a new version comes out.
-
Previous score and scorecard:
F (poor)
#431 (closed) -
New Benchmark score:
F (poor)
- User interview conclusions: https://gitlab.com/gitlab-org/uxr_insights/issues/917
- Recommendations: #516 (closed)
- Walkthrough video: https://youtu.be/8Z7P3Q-m0KM
- Slides: Google slides
UX Scorecard Checklist
Click me to collapse/fold the scorecard checklist.
Learn more about UX Scorecards
-
Add this issue to the stage group epic for the corresponding quarter's UX scorecards. -
After working with your PM to identify a top job, write it using the Job to Be Done (JTBD) format: When [situation], I want to [motivation], so I can [expected outcome]. Review with your manager to ensure your JTBD is written at the appropriate level. Remember, a JTBD is not a user story and should not lead directly to a solution. -
Make note of which personas might be performing the job, and link to them from this issue's description. Keeping personas in mind allows us to make the best decisions to address specific problems and pain points. Note: Do not include a persona in your JTBD format, as multiple types of users may complete the same job. -
If your JTBD spans more than one stage group, that’s great! Review your JTBD with a designer from that stage group for accuracy. -
Review the current experience, noting where you expect a user's high and low points to be. Capture the screens and jot down observations. - If you're re-scoring the experience, recapture the entire flow. You will likely have some of the artifacts (i.e. a UI screen that wasn't changed) that you can simply reuse.
-
It's also advised that you ask an actual user to accomplish the JTBD. Record this session and document their experience. Note that 3-5 additional users are preferred, as this provides valuable insights and removes subjectivity. Make sure to avoid setting up a task-based usability study. The goal is to provide the participant context (the JTBD) and listen and watch how they attempt to complete the job. What we learn may differ from participant to participant. -
Using what you learned in the previous steps, apply the following Emotional Grading Scale to document how a user likely feels at each step of the workflow. Add this documentation to this issue's description. - Positive: The user’s experience included a pleasant surprise — something they were not expecting to see. The user enjoyed the experience on the screen and could complete the task, effortlessly moving forward without having to stop and reassess their workflow. Emotion(s): Happy, Motivated, Possibly Surprised
- Neutral: The user’s expectations were met. Each action provided the basic expected response from the UI, so that the user could complete the task and move forward. Emotion(s): Indifferent
- Negative: The user did not receive the results they were expecting. There may be bugs, roadblocks, or confusion about what to click on that prevents the user from completing the task. Maybe they even needed to find an alternative method to achieve their goal. Emotion(s): Angry, Frustrated, Confused, Annoyed
-
Use the Grading Rubric to provide an overall measurement that becomes the Benchmark score for the experience (one grade per JTBD), and add it to this issue's description. Document the score in the UX Scorecard Spreadsheet. -
Once you’re clear about the user’s path, create a walkthrough video that documents the existing experience. Begin the video with a contextual introduction including: your role, stage group, and a short introduction to your JTBD and purpose of the UX scorecard. This is not a "how to" video, but instead should help build empathy for users by clearly showing areas of potential frustration and confusion. (You can point out where the experience is positive, too.) The Emotional Grading Scale you documented earlier will help identify areas to call out. At the end of the video, make sure to include narration of the Benchmark Score. Examples here and here.-
If you're re-scoring the experience, walkthrough the entire flow again. For narration, you can highlight the recent improvements but still call out any areas that could still use some tweaking (in the next round of iterations, if applicable). The re-score video, in theory, should be shorter since we've hopefully eliminated a few bumps in the user flow.
-
-
Post your video to the GitLab Unfiltered YouTube channel, and link to it from this issue's description. -
Link to your video in the Engineering Week in Review. -
Create a recommendation issue for this JTBD and add it to the same stage group epic as this issue. -
Following the UX Scorecards setup instructions, create an issue (and epic, if needed) to rescore the same JTBD the following quarter to see if we have made improvements. We will use the grades to monitor progress toward improving the overall quality of our user experience. Add that issue as related to this issue.
Rescoring
Based on heuristics evaluation, the proposed recommendations #505 (closed) focused the following painpoints:
- Creating and managing a release is currently only possible through the Releases API.
- Releases are created via Tags, but there’s no clear differentiation between lightweight versus annotated tags.
- Inability to track project status at a high level within GitLab’s interface.
- Releases in GitLab are not yet well associated with other data that exists in the single application.
- Managing releases in GitLab from a traditional, manual or semi-automated release checklist point of view is difficult. We create manual tasks and check them off as we go.
The following improvements were made based on the Recommendations:
- Make it possible to edit a Release via UI gitlab#26016 (closed)
- Improve the UI of the Releases page (tackled with gitlab#26016 (closed))
- Allow milestone(s) to be associated with a release: gitlab#29020 (closed)
- Show issue summary on Release page gitlab#31289 (closed)
- Notifications for Releases gitlab#26001 (closed)
- Links to SHA commits in release notes don’t display in the new Releases page (but do display on the Tags page) gitlab#28325 (closed)
- Add more Release artifact types gitlab#26014 (closed)
- Add pagination to Releases page gitlab#14955 (closed)
Heuristics analysis
SEE MURAL BOARD WITH HEURISTIC ANALYSIS
User interviews
New score
F (Poor)
: Workflow leaves user confused and with no direction of where to go next. Can sometimes cause the user to go around in circles or reach a dead end. Very high risk of abandonment, and user will most likely seek other methods to complete the task.
- Frustration: Very High
- Task Completion: Very Unlikely
- Steps to Complete Task: Lacking
Next steps
Recommendations for Releases stay as is. The recommended focus is on being able to create a release via UI gitlab#32812 (closed), supporting all release data on the release form &2312 (closed), and creating a dedicated page for each release entry gitlab#32827 (closed).
See #505 (closed) for full recommendations.