Skip to content

Make User Experience the primary focus for rest of the CY2020.

Proposal

Take rest of Q3 (i.e. 13.5) and Q4 and prioritize User Experience as our primary focus area across the product.

Why

Scott raised an issue to help us shift from a highly effective feature machine to an equally effective, if not more, product team that can effectively drive adoption and monetization.

In fact, there is recent evidence that our feature adoption may be hampered due to our user experience. UX Research team attributes settings, navigation and UI Polish as the primary drivers for declining SUS scores over the past 6 quarters.

Furthermore, there are areas within the product where user experience can be improved outside of settings, navigation, and UI Polish which will help drive adoption and monetization. Examples include improving error messaging (e.g., without next steps, when nothing happened (example which is now fixed)), monolith issues slowing down performance, GitLab scanner results, and improving integrations. Here is an example of a very public comment related to these sorts of issues.

There is a strong belief internally that a superior user experience is one of the primary levers to drive that.

Market agrees that UX can be the key differentiator

There is plenty of market evidence to support this claim.

  • Top-quartile MDI (McKinsey Design Index) scorers increased their revenues and total returns to shareholders (TRS) substantially faster than their industry counterparts did over a five-year period—32 percentage points higher revenue growth and 56 percentage points higher TRS growth for the period as a whole.
  • The results held true in all three of the industries we looked at: medical technology, consumer goods, and retail banking. This suggests that good design matters whether your company focuses on physical goods, digital products, services, or some combination of these.
  • TRS and revenue differences between the fourth, third, and second quartiles were marginal. In other words, the market disproportionately rewarded companies that truly stood out from the crowd.

This is an intuitively agreeable stance. However it is (in the context of GitLab) an untested hypothesis that user experience is a key to unlock adoption.

We all know we can do better

Anecdotally there is plenty of evidence inside GitLab that we ourselves have often shied away from dog-fooding or using features that we call minimal or viable. And we find a lot of places where we can make improvements. Infact, while I was writing this issue up, I got an email and found one area that could be [improved](gitlab-org/gitlab#238027. I've realized that I've started to ignore these instead of raising these. I plan to do better.

But we have never put this hypothesis to test ...

Having said that, it is very hard to test it for GitLab. We don’t use A/B/n experimentation frameworks. We have just started on our metrics journey, but there is a long way to go.

If we took rest of the CY 2020 and tested it...

If each team across the board identified and prioritized UX improvements, we will have created a reasonably good approximation to validate our hypothesis that UX will help drive adoption.

  • At worse, we would have made amazing improvements to User experience that will benefit tens of millions of our users. We will also feel more pride in our work and make our own internal GitLab usage experience more pleasant and probably more efficient. In addition, if our hypothesis turns out to be false, we would have fallen behind known competitors on features that will prevent us from winning more competitive opportunities. However, if that happened, we would have established that our hypothesis was false (i.e. UX is a major driver for GitLab adoption and monetization).

One more con - not product related, but related to team health is included here

There had been some fairly recent pushback from PMs who felt like their DRI-ness was being taken away by the combo of forced SLAs, Efficiency Projects and Top IACV drivers so this is further eroding that sense of autonomy.

  • At best - We would have unlocked a new adoption driver and learnt what works and what doesn't.

I like the idea but… the devil is in the details

We’ve already updated the handbook to make UX one of the highest priorities, over IACV drivers, feedback from dogfooding, feature velocity and predictability.

In there the most important area of judgement would be between IACV and UX.

Edited by David DeSanto