[VSA] Unclear that metrics are medians

Summary

I also am having a hard time reconciling our Mean Time To Merge KPI [of about 4 days] with https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics, which indicates the Review time is about 23 hours. (from #21863 (comment 312272174))

The definitions seem to capture the same thing:

  • MTTR is: "how long it takes from submitting code to being merged".
  • VSA says the Review stage is: "Time between merge request creation and merge/close"

Is the MTTR time too high or is the VSA Review time too low?

Steps to reproduce

  1. Visit https://gitlab.com/groups/gitlab-org/-/analytics/value_stream_analytics
  2. Notice the "Review" time is much shorter than MTTR.

What is the current bug behavior?

The VSA "Review" stage metric is lower than expected, at least for the gitlab project.

What is the expected correct behavior?

The VSA "Review" stage metric is accurate, and similar to MTTR.

Relevant logs and/or screenshots

MTTR (seems to be accurate): Screen_Shot_2020-04-01_at_3.36.42_PM

VSA Review (seems to be too low): Screen_Shot_2020-04-01_at_3.37.07_PM

Output of checks

This bug happens on GitLab.com.

Additional details

The MTTR query from Sisense:

SELECT 
  mrrs.merge_month
  , AVG(days_from_review_to_merge) as average_days_review_to_merge
  , approx_percentile(days_from_review_to_merge, 0.95) as "95th percentile days review to merge"
  , AVG(days_to_review) as average_days_to_review
  , approx_percentile(days_to_review, 0.95) as "95th percentile days to review"
  , MAX(dev_product_mrs_merged_per_dev_team_member) as "MR Rate"
  , 4.21 as "Target"
FROM merge_requests_review_summary mrrs
WHERE [mrrs.merge_month=daterange]
GROUP BY 1
Edited by Dan Jensen