Northstar metric Verify: Testing
Purpose
The product team is defining Northstar metrics for all groups to help guide how we make decisions. This issue captures the work and collaboration required to define and instrument the North star metric for grouptesting
Northstar metric
TOP METRIC: Count of active paid testing feature users
The sum of unique users who have interacted with a paid testing feature
This could should include:
- Web Performance Report
-
Can we track self-managed? Waiting until Snowplow is available in 2021 -
Are we already tracking SaaS? No -
Which action in the app is "usage" - Click to open the MR widget
-
- Full Code Quality Report
-
Can we track self-managed? Waiting until Snowplow is available in 2021 -
Are we already tracking SaaS? No -
Which action in the app is "usage" - View the code quality tab - /-/pipelines//codequality_report
-
- Custom metrics
-
Can we track self-managed? Waiting until Snowplow is available in 2021 -
Are we already tracking SaaS? Creation yes, access no -
Which action in the app is "usage" - Click to open the MR widget
-
- Group Code Coverage page (planned functionality)
-
Can we track self-managed? Waiting until Snowplow is available in 2021 -
Are we already tracking SaaS? No -
Which action in the app is "usage" - Defined in &2838 (closed)
-
- Visual Review Tools
-
Can we track self-managed? Waiting until Snowplow is available in 2021 -
Are we already tracking SaaS? Yes Gitlab::Tracking.event('Notes::CreateService', 'execute', { label: 'anonymous_visual_review_note', value: note.id })
-
Which action in the app is "usage" - Leaving a comment
-
Plan
- Figure out what metrics are available today and where the data is coming from so we are clear what is only .com, self-hosted and both.
- As @rickywiens noted in a comment completing these items should answer all the questions above.
-
Determine if there is a reason why we cannot includeNot useful for Stage Monthly Active Paid Users SMPAUSELECT COUNT(*), file_type FROM ci_job_artifacts GROUP BY file_type;
or some other in order to count the total job artifacts by type. We should co-ordinate with the team responsible for usage ping to see what they think -
Determine the lifecycle of rows in the ci_job_artifacts tableNot useful for Stage Monthly Active Paid Users SMPAU
- Document what exists today in this issue
- Document remaining work in the parent epic.
What does success look like, and how can we measure that?
The Testing Metrics dashboard is updated to include a chart showing unique pipelines per week that generated at least one piece of testing data. Not useful for Stage Monthly Active Paid Users SMPAU
A new dashboard can be created counting Stage Monthly Active Paid Users SMPAU over time.
Links / reference
Edited by drew stachon