FY22 Q4 UXR KR1: Building a database for SUS verbatim findings

Background

As we acquire more SUS data quarter after quarter, there’s been scenarios where we’ve had to look at that data across quarters for verbatim trends, data points, and supporting quotes. Such data is used to support KR-related work and experience improvements.

The Problem

While it's possible to do gather the data across quarters, it's inefficient and time-consuming; as the data exists in multiple Google Sheets (one for each quarter) - and the act of looking for specific data across multiple Google Sheets is laborious and error-prone.

The Solution

Build a single database that stores all of the historical and future SUS data. While this database won't take the place of the Google Sheets, it serves the purpose of accessing past data across quarters easily and efficiently.

Scope

The scope of this iteration will be limited to SUS verbatim and attributes associated with each verbatim. Future iterations could explore including the individual SUS questions and emotions derived from data science.

STRETCH GOAL:

  • Include Overall SUS score and SUS score by question. This will enable us to look at trends on a per question basis over time.
  • Make database entries amendable

Requirements

  1. To meet the use case outlined above, the database shall contain the following fields for a given row/entry:

    • Fiscal Year
    • Quarter
    • SaaS or Self Managed
    • Verbatim
    • SUS score
    • SUS theme
    • New or Mature user
    • Plan Type
    • Stage (when applicable)

  2. The information in the Google Sheets should be identical to the information in the database.

  3. The database charts (tables) should be sortable, filterable, and searchable. Example scenarios the database experience must support:

    • Finding verbatim associated with the Performance SUS theme during FY22 Q1 and FY22 Q3, SaaS only
    • Surfacing verbatim from respondents who had a SUS score of 35 or lower, across all quarters, SaaS and Self Managed
    • Finding SUS scores and verbatim for New and Free users
    • Surfacing SUS verbatim for a specific Stage within FY22 Q3 for SaaS and Self Managed
    • Sending a link to the database that has a pre-determined sort, filter, and/or search
    • etc

  4. The database is easy to use.

  5. The database can accommodate future columns (ex: emotion)

  6. The database is accessible by anyone within GitLab

  7. Someone can easily send a receive a link to the database that shows a pre-determined sort, filter, and/or search

  8. The database should not contain PII (Personal Identifiable Information) about respondents

Steps

  • 2021-11-05: @moliver28 Reach out to the data team to get their guidance, based on our objectives (ex: What's the best approach? Do we already have this for free with Sisense?, etc)
  • 2021-11-10: @moliver28 Investigate to understand where the database will reside and if it needs SAFE classification
  • 2021-11-20: @moliver28 Define and build database structure
  • 2021-12-01: @NickHertz Import data cleanly, accurately, and efficiently
  • 2021-12-05: @NickHertz Prove that importing future data is also clean, accurate, and efficient
  • 2021-12-10: @NickHertz Test the above scenarios, along with other related scenarios
  • 2021-12-16: @moliver28 Create a new handbook page with video demoing how to use it
  • 2021-12-29: @moliver28 Publicly announce the existence of the database
Edited by Adam Smolinski