Category Maturity Scorecard - Monitor:APM FY21-Q1 - Logging CM Validation
- Research issue: Validating Logging Maturity
- Previous score and scorecard: N/A
- Walkthrough: Mural link
- Recommendations: Experience recommendations
Category Maturity Scorecard Checklist
Learn more about Category Maturity Scorecards
-
Review the Category Maturity Scorecard handbook page and follow the process as described. Reach out to the UX Researcher for your stage if you have questions. -
Document the results of each participant's session using the CM Scorecard Results Template -
Add Zoom links for each participants' session recordings. These can be found in the Recordings section of the Zoom web UI. -
If the participant has not granted permission to share the recording publicly, ensure the sharing settings are set to GitLab-only. -
Summarize the results and give the final grade using this template. -
Create a recommendation issue for these sessions.
CM Scorecard Results Template
Jobs to be done
When troubleshooting or verifying the status of my services I want to easily find the necessary information within my logs, so I can quickly get the system back up and running.
Scenario
Imagine you work as a DevOps engineer for a software company. There’s been an incident while you were out of office, so you go to the incident issue to see what happened. Your goal is to find the logs that correspond to the time of the incident. Your application runs in the "logging-user-testing" environment.
Participants
Hassan
Background
DevOps engineer, reposnible for DevOps workflow, uses GitLab daily.
Scenario evaluation
- Ease of completion (pre-task): Not easy or difficult
- Ease of completion (post-task): Somewhat easy (its clearer that what he expected)
- Task completion confidence: -
- Meaningful clicks: -
- Heuristic evaluation: Successfully, but with some unnecessary steps or difficulty
Ken
Background
Senior Engineer. In DevOps for 7 years. Used GitLab privately for 5 years.
Scenario evaluation
- Ease of completion (pre-task): Very easy
- Ease of completion (post-task): Somewhat difficult
- Task completion confidence: Absolutely confident
- Meaningful clicks: -
- Heuristic evaluation (How well did the participant complete the scenario?): Successfully, but with some unnecessary steps or difficulty
Other notes
Overall grade: Not assigned
Note: Category Maturity Scorecard work uncovered a critical usability problem. We decided to fix it before continuing with further work. Based on the data from 2 sessions that we conducted, there is a chance that we would get a passing grade of C for Viable.
This grade represents the average of the ratings for all participants based on the grading rubric