Skip to content

Solution validation: Pajamas Design System sentiment

What did we learn?

Results
  • Design system consumers liked that the system allows them to focus on user problems 
  • Accessing the design system through three different platforms causes usability issues and mistakes
  • There are many documented gaps (todos) and undocumented gaps (spacing) reducing consumer trust and efficiency
  • Full contributions require significant effort resulting in incomplete contributions or avoiding them completely
  • Differences in Vue, HAML and bootstrap-vue create additional effort to understand and implement components

What's this issue all about? (Background and context)

After joining groupfoundations I had questions about how we know if our design system is working, and what is the usability of the system. I considered if Pajamas should have its own SUS, and how we could perform continuous interviews.

I recently re-read How to understand your Design System’s health which talks through a process followed at Deliveroo.

I thought many of the situations sounded similar and wondered if there would be value in following some of the same steps.

Included is a quantitive study to establish a sentiment baseline. This issue is to explore creating something similar (or using the same survey) to establish a baseline for Pajamas, and track changes over time. Having these metrics would allow us to measure the outcomes of our efforts.

What are the overarching goals for the research?

  • Create a quantitative survey that we can use to gauge sentiment.
  • Learn the current sentiment, by recording a baseline 'score' for Pajamas.
  • Identify outcomes and trends by repeating the survey at intervals.

What hypotheses and/or assumptions do you have?

No assumptions right now. Just looking for a baseline.

What research questions are you trying to answer?

Is the design system working? Or to put it another way, does the design system:

  • improve the quality and speed of building GitLab?
  • have enough depth and breadth for our consumers?
  • make consumers feel confident, and enable them to contribute back?

We want to see if these are changing (getting better) over time as we continue to work on Pajamas.

What persona, persona segment, or customer type experiences the problem most acutely?

  • Product Designers @ GitLab
  • Frontend engineers @ GitLab
  • Product Managers without a stage group designer @ GitLab

I expect product designers, frontend engineers, and product managers are the design systems consumers we are most interested in.

We are focused on GitLab team members contributing directly to the GitLab product.

What business decisions will be made based on this information?

I imagine the results of this will help us prioritise and schedule work. It might also provide us with a metric we can measure as part of goals or OKRs.

What, if any, relevant prior research already exists?

Who will be leading the research?

@danmh with support from @chrismicek. I imagine repeated surveys would be lead by groupfoundations .

What timescales do you have in mind for the research?

Next 1–3 milestones.

Relevant links (script, prototype, notes, etc.)

We could run some pilot interviews to identify our own themes. Or we could use the same survey Deliveroo used as I think there is significant overlap. I've edited it slightly to closer match our context, but left the original author's 'why' there for reference.

Script (Google Doc)

Form (edit)

Form (responder version)

Edited by Dan MH