Category Maturity Scorecard - Monitor Q1 2020 - Error Tracking CM Validation
- Research issue: ux-research#651 (closed)
- Previous score and scorecard: N/A
- Walkthrough: Mural link
- Recommendations: gitlab#213418 and gitlab#213413
Category Maturity Scorecard Checklist
Learn more about Category Maturity Scorecards
-
Review the Category Maturity Scorecard handbook page and follow the process as described. Reach out to the UX Researcher for your stage if you have questions. -
Document the results of each participant's session using the CM Scorecard Results Template -
Add Zoom links for each participants' session recordings. These can be found in the Recordings section of the Zoom web UI. -
If the participant has not granted permission to share the recording publicly, ensure the sharing settings are set to GitLab-only. -
Summarize the results and give the final grade using this template. -
Create a recommendation issue for these sessions.
CM Scorecard Results Template
Jobs to be done
Primary JTBD:
When I'm proactively reviewing the list of errors thrown by my code, I want to be able to escalate investigation of the most serious errors, so I can make sure they are addressed before they impact customers.
Secondary JTBD:
When I'm proactively reviewing the list of errors thrown by my code, I want clean up the list of errors, so that I'm only seeing the errors that are important to me.
The assumption is that we'll be testing the primary JTBD as part of this research effort.
Scenarios
Scenario N
Imagine you work as a DevOps engineer for a company that sells flight tickets online. It’s Monday morning, and you are starting your day by going through and reviewing errors that occurred over the weekend.
Participants
Kunal
- Uses GitLab for Error Tracking and for Security/Compliance
- Actively uses our Error Tracking workflow!
- Successfully completed all tasks
Marc
- Uses GitLab, but not error tracking
- Successfully completed all tasks
Sachin
- Successfully completed the tasks, but had low confidence that he was completing them properly
- Hadn't used GitLab issues before, so was confused by things like "move issue"
- Wasn't sure if ignoring the errors ignored them for everyone or just for him. Wanted to ignore for a specific time
Brady
- Knew a ton about GitLab
- Easily completed the tasks
- Was a little unsure about how issue weights impacted errors
Josue
- Successfully completed the tasks.
- Wasn't super confident he completed them correctly, but difficulty seemed to stem from the way the tasks were phrased (example, what does "escalate" actually mean in GitLab)
Scenario Notes
Other notes
{{Add any additional notes that came from freeform discussion or elsewhere}}
Overall Grade: C
This grade represents the average of the ratings for all participants based on the grading rubric