<!--Please answer the below questions to the best of your ability.--> ## Background and context One of the open questions from https://gitlab.com/gitlab-org/ux-research/-/issues/1374 was: > How do users grade the product performance? How does it compare to our competitors? This research aims to answer this question by setting a baseline for **user's perceived performance of <abbr title="merge request">MR</abbr>s** and benchmark it against our top competitors. ## Details - **Overarching goals for the research**: Set a baseline for **user's perceived performance of <abbr title="merge request">MR</abbr>s** and benchmark it against our top competitors. - **Hypotheses and/or assumptions**: User's perceived performance of MRs is worse compared to our top competitors. This assumption comes from the research synthesis done in https://gitlab.com/gitlab-org/ux-research/-/issues/1374. - **Research questions**: - How do users grade the product performance of MRs? - How does it compare to our competitors? - **Persona that experiences the problem most acutely**: [Sasha, Software Developer](https://about.gitlab.com/handbook/marketing/strategic-marketing/roles-personas/#sasha-software-developer) - **Business decisions to be made based on this information**: Setting a baseline and benchmark will allow us to measure the ROI of improvements that affect this perception of performance over time. - **Prior research**: https://gitlab.com/gitlab-org/ux-research/-/issues/1374 - **Research lead**: @pedroms - **Research timescale**: 2 weeks ## Metrics Together, the following metrics help build an holistic picture of how users perceive MR performance. 1. :robot: **Machine reported**: We'll focus on the [Largest Contentful Paint](https://web.dev/lcp) (LCP) metric, that we already track today in our GitLab.com and test instance Sitespeed tests — see https://gitlab.com/gitlab-org/ux-research/-/issues/1473 for details. For competitors, rely on https://forgeperf.org. 1. :bust_in_silhouette: **Human reported** (GitLab only): In-product survey invite (specific to MRs) to get in-context responses that mitigate biases ([HaTS](https://notesforgrowth.github.io/HaTS/) method matured at Google). For competitors it's challenging to get reliable data so we won’t pursue this at this time. 1. :runner: **Task times**: Predict task times using the [KLM](https://measuringu.com/predicted-times/) method. We can predict task times for expert users with 80%–90% confidence. ## Relevant links (script, prototype, notes, etc.) - https://developer.mozilla.org/en-US/docs/Glossary/Perceived_performance - https://www.youtube.com/watch?v=7ubJzEi3HuA - https://quod.lib.umich.edu/w/weave/12535642.0001.403?view=text;rgn=main - https://marvelapp.com/blog/a-designers-guide-to-perceived-performance/ - https://www.smashingmagazine.com/2015/09/why-performance-matters-the-perception-of-time/ - https://www.smashingmagazine.com/2015/11/why-performance-matters-part-2-perception-management/ - https://www.smashingmagazine.com/2015/12/performance-matters-part-3-tolerance-management/#psychological-time-tolerance-management - https://blog.teamtreehouse.com/perceived-performance - https://blog.codinghorror.com/actual-performance-perceived-performance/ - https://www.nngroup.com/articles/website-response-times/ - https://forgeperf.org/ - https://www.chrisharrison.net/index.php/Research/ProgressBars2 <!-- #### TODO Checklist Consider adding a checklist in order to keep track of what stage the research is up to. Some possible checklist templates are here: https://about.gitlab.com/handbook/engineering/ux/ux-research-training/templates-resources-for-research-studies/#checklists -->