Skip to content

Usability testing

References

There are many books and other online resources available regarding usability and usability testing. Folks should communicate with the UX team if they want to learn more. A lot of the following is informed by the research of Steve Krug.

Usability tests and process

  • The purpose of a usability test is to reveal major usability problems in a piece of software.
  • Typically, a round of tests focuses on a handful of features.
  • Usually, up to 5 users (and thus 5 tests) are sufficient to determine the major problems.
  • The goal is not to find an exhaustive list of problems. Rather, it is to identify a small number of major ones.
  • The tester is given a list of tasks to perform, with minimal guidance and background. They then perform the test, talking through what's going on in their head.
  • The tester does not necessarily need to have a lot of expertise in the specific software or even domain. Often, usability problems will still be easily detected. Furthermore, the software should be intuitive for a user to perform basic tasks with minimal explanation.
  • Nonetheless, we should still have some diversity of users to reveal more problems. Demographic diversity, expertise and domain diversity, etc.
  • People creating the software observe and take notes.
  • Modern tools allow us to automate and outsource a lot of this process. (See below.)
  • Afterward, the design, engineering, and product teams should determine which major problems to address, based on the usual product development process. Usability tests are just another input of ideas to our process.
  • We should have usability tests approximately once a month. That's not too frequent, and not too long between tests, and aligns well with GitLab's monthly releases.
  • Usability testing is not prototype testing. It tests for usability problems in existing features and the software as it stands now.
  • It is a powerful tool since research indicates that only a small number of tests will reveal the major problems.

Executed tests

These tests were created using the testing platform usertesting.com.

December 13, 2016

Scenario: Business manager at a bank working with a software team to create a mobile app

YouTube

Survey

  • What frustrated you most about this site?
    • I felt like I needed to learn a lot to understand it.
  • If you had a magic wand, how would you improve this site?
    • I would improve it by making it attractive and more appealing to the general public.
  • What did you like about the site?
    • I liked the purpose of the site and it really interested me.
  • How likely are you to recommend this site to a friend or colleague (0=Not at all likely, and 10=Very Likely)?
    • 7