Group Level Analytics
Problem to solve
During customer conversations we have found that engineering and product teams can span multiple groups and projects. At the moment, most of our analytics are on a project level, which makes it very difficult for an EM / PM / Executive to get a sense of where the majority of effort is spent and understand cycle times and bottlenecks across teams (we have received similar feedback from @dhavens). Moreover, our analytics are dispersed across the tool and are not particularly easy to find. We have:
- Language statistics on a project level: https://gitlab.com/gitlab-org/gitlab-ce/graphs/master/charts
- 'Contributor' Analytics on a project + branch level: https://gitlab.com/gitlab-org/gitlab-ce/graphs/master
- Contribution Analytics on a group level: https://gitlab.com/groups/gitlab-org/-/analytics
- CI/CD charts on a project level: https://gitlab.com/gitlab-org/gitlab-ce/pipelines/charts
- Cycle Analytics on a project level: https://gitlab.com/gitlab-org/gitlab-ce/cycle_analytics
- Insights on a group + project level: https://gitlab.com/gitlab-org/gitlab-ce/insights
- Coverage on a project level: https://gitlab.com/gitlab-org/gitlab-ee
- Operations metrics /dashboard on a group + project level (with a somehow different role): https://gitlab.com/gitlab-org/gitlab-ce/environments/11480/metrics, https://gitlab.com/-/operations?nav_source=navbar
- Issue analytics on a group level: https://gitlab.com/groups/gitlab-org/-/issues_analytics
- Milestone analytics on a group level: https://gitlab.com/groups/gitlab-org/-/milestones/28
Most of these analytics do not contribute to any actionable insights since they are at MVC stage and difficult to connect.
For example, executives would like to be able to have a single view of languages used across the instance (ex: https://gitlab.com/gitlab-org/gitlab-ce/issues/62294), compare productivity and velocity across teams, groups, projects, etc. Currently, managers at different levels of the organization are looking at different information, which is often inconsistently defined between teams and difficult to aggregate.
Intended users
Development Team Lead, Product Manager, Executive
Further details
Implementing analytics at an instance level will allow us to move forward with our vision for VSM and have one place for dashboards, which can be used by developers, managers and executives. Many customers have expressed interest in instance level analytics with customizable dashboards allowing users to drill-down in order to uncover actionable insights. (https://gitlab.com/gitlab-org/gitlab-ce/issues/48807)
Proposal
-
Given the size of instances like GitLab.com, our MVC here will be starting analytics on a group level.
-
A logged-in PM/EM, who is currently on an instance page, should have an easy navigation to an analytics page. @cperessini, after discussing with Eric and Jeremy, I think it's better to have it as
Analytics
on the top nav bar, just before the Ops Dashboard icon. We can work on combining with the admin dashboard at a later date. We can actually remove Snippets as no-one uses it and I am not aware of any marketing effort to make it an app important feature deserving space on the nav bar? -
Given we are putting
Analytics
on the Nav Bar (in order not to confuse use with discoverability given the feature will move to instance level), let's have an easy way for a user to select a group he is interested in. We can potentially use our existing dropdown for Groups or the selection of projects feature on the Ops Dashboard for that? (cc: @cperessini, what do you think?) -
Once the user clicks on 'Analytics', a page similar to the below would open and on the left we can have:
Productivity Analytics
andCycle Analytics
. Each page will have different charts that a user can save in the future as his private view (ex: https://gitlab.com/gitlab-org/gitlab-ee/issues/6019, https://gitlab.com/gitlab-org/gitlab-ce/issues/58435)
- On the top of each page, we should have a 'global' dashboard filter by group (in the future, for the MVC not), project, milestone, author, assignee (as per your proposal here: https://gitlab.com/gitlab-org/gitlab-ce/issues/48807). I would suggest to have the data range filter separate on the top right like in your proposal but make it an actual calendar range, where people can select the timeframe with a date picker like this one (http://www.daterangepicker.com). The reason for that is that many companies want to look at longer timeframes as they don't release more often than once a month.
https://gitlab.com/gitlab-org/gitlab-ce/issues/29359 https://gitlab.com/gitlab-org/gitlab-ce/issues/28123
Old issuesPermissions and Security
The dashboard should inherit the project/ groups permissions of users, i.e. I can only see an aggregation of subgroups, projects I have access to.
A user should have at least reporter access in order to see cycle analytics.
Documentation
Testing
What does success look like, and how can we measure that?
Measure: We should include the page in user ping to start with and measure how many people visit and how long they stay on it.
Success: As we develop this further, we expect users to spend a significant time deep diving on the page at least once a week.