Productivity Analytics MVC
Problem to solve
The development cycle is currently a blackbox for many companies. Value stream mapping is difficult to do well. Even when processes are mapped, drilling down in a systematic way into the issues that are slowing the team down is nearly impossible to do. By deep diving into productivity analytics, an EM can understand how well they are deploying their engineering resources and spot patterns across individuals and projects. Productivity can slow down for many reasons ranging from degrading code base to quickly growing teams. Hence, deep diving into productivity analytics first will likely already convey important problematic or exemplary patterns across individuals and teams.
Intended users
EMs
JTBD:
When I’m analyzing the productivity of my team (situation), I want to recognise problems and patterns that delay MRs from being merged (motivation), so I can help speed things up and prevent it from happening in the future (expected outcome).
Further details
We would like to understand the time required to approve a merge request, i.e. from first commit until approval and what are the patterns that cause us to slowdown. Ideally, one would expect smaller, regular commits since it's easier for people to review and is likely to generate conflicts.
Proposal
An engineering manager should be able to come to a dashboard where they can filter by any of our global filters (gitlab-ce#62346 (closed)), i.e. group, subgroup, project, date-picker, author, milestone, assignee, label.
-
Let's create a histogram (all charts should be rendered based on the global search on the page) which plots the a)
Time taken to merge an MR
. A user should be able to select either one or multiple bars, which would be used for subsequent graphs. We will only look at merged MRs to start with. -
Let's create another histogram which is rendered based on the selection in the first and has a dropdown of the following series:
- b) Time from first commit until first comment
- c) Time from first comment to last commit
- d) Time from last commit to merge. This is out first attempt to start breaking down the MR work to get to a stage where we can represent
touch time
vswait time
.
- Let's create a third histogram which is rendered based on the selection in the first and has a dropdown of the following series
- Number of commits per MR
- Number of LOCs per commit
- Number of files touched
- We should be able to see a list of MR's based on the selection in a way similar to here, but we should have all columns of the data, i.e. a), b), c), d) and users should be able to order by these columns.
Let's add a scatterplot with a trendline for the median as described here (https://gitlab.com/gitlab-org/gitlab-ce/issues/45304) as well, where a user can select any of the above series (Time taken to merge an MR, Time from first commit until first comment, Time from first comment to last commit, Time from last commit to merge). The x -axis can be the date the MR is merged, y axis - N of hours / days. (@pshutsin, this has a similar example of a chart: &228 (closed))
@pshutsin - as discussed, in terms of data:
- let's have a background migration for 6m of data when we deploy the feature/ customer updates version
- have the possibility for an admin to return data with a different timespan
Solution
Empty state | Group selected | Chart bar selected | Timeframe dropdown with datepicker | Timeframe dropdown |
---|---|---|---|---|
Illustration for the empty state: prod-analytics-empty-state.svg
With the MVC solution, we're solving half of the problem as outlined with our JTBD:
When I’m analyzing the productivity of my team (situation), I want to recognise problems and patterns that delay MRs from being merged (motivation), so I can help speed things up and prevent it from happening in the future (expected outcome).
We're not yet solving the problem of fining MRs that are still open and possibly stuck. We'll do that in future iterations.
Key interactions:
* the user can drill down by selecting values in any of the 3 charts on top @matejlatin - we decided to have only the top chart with filtering functionality for the MVC, which I think as better given the concerns with the hierarchical filtering functionality and we'll think through a better way to filter for the different metrics. For now, we are just thinking of adding a sentence under the Merge Request histogram saying Please select columns to filter all subsequent charts
.
- as soon as a value in one chart is selected, all the information that comes after it is refreshed.
- the user can unselect a value in the chart by clicking on it again
Mobile designs
Empty state | Group selected | Alt table design | Timeframe dropdown with datepicker | Timeframe dropdown |
---|---|---|---|---|
Permissions and Security
The dashboard should inherit the project/ groups permissions of users.
Documentation
Testing
What does success look like, and how can we measure that?
Measure: We should include the page in user ping to start with and measure how many people visit and how long they stay on it.
Success: As we develop this further, we expect users to spend a significant time deep diving on the page at least once a week.