Category Maturity Scorecard - Monitor:APM FY21-Q1 - Logging CM Validation

Category Maturity Scorecard Checklist

Learn more about Category Maturity Scorecards

  1. Review the Category Maturity Scorecard handbook page and follow the process as described. Reach out to the UX Researcher for your stage if you have questions.
  2. Document the results of each participant's session using the CM Scorecard Results Template
  3. Add Zoom links for each participants' session recordings. These can be found in the Recordings section of the Zoom web UI.
  4. If the participant has not granted permission to share the recording publicly, ensure the sharing settings are set to GitLab-only.
  5. Summarize the results and give the final grade using this template.
  6. Create a recommendation issue for these sessions.

CM Scorecard Results Template

Jobs to be done

When troubleshooting or verifying the status of my services I want to easily find the necessary information within my logs, so I can quickly get the system back up and running.

Scenario

Imagine you work as a DevOps engineer for a software company. There’s been an incident while you were out of office, so you go to the incident issue to see what happened. Your goal is to find the logs that correspond to the time of the incident. Your application runs in the "logging-user-testing" environment.

Participants

Hassan

Background

DevOps engineer, reposnible for DevOps workflow, uses GitLab daily.

Interview recording link

Scenario evaluation

  • Ease of completion (pre-task): Not easy or difficult
  • Ease of completion (post-task): Somewhat easy (its clearer that what he expected)
  • Task completion confidence: -
  • Meaningful clicks: -
  • Heuristic evaluation: Successfully, but with some unnecessary steps or difficulty

Ken

Background

Senior Engineer. In DevOps for 7 years. Used GitLab privately for 5 years.

Interview recording link

Scenario evaluation

  • Ease of completion (pre-task): Very easy
  • Ease of completion (post-task): Somewhat difficult
  • Task completion confidence: Absolutely confident
  • Meaningful clicks: -
  • Heuristic evaluation (How well did the participant complete the scenario?): Successfully, but with some unnecessary steps or difficulty

Other notes

Meeting notes

Experience recommendations

Overall grade: Not assigned

Note: Category Maturity Scorecard work uncovered a critical usability problem. We decided to fix it before continuing with further work. Based on the data from 2 sessions that we conducted, there is a chance that we would get a passing grade of C for Viable.

This grade represents the average of the ratings for all participants based on the grading rubric

Edited by Anoop Dawar