FY20-Q2 Quality Department OKR: Increase maturity of Quality Department KPIs => 80%
-
Key Result: Monthly new bugs per stage group. gitlab-org/gitlab-insights#108 (closed)
=> 100%
-
Key Result: Mean time to resolve S1-S2 functional defects. gitlab-org/gitlab-insights#109 (closed)
=> 100%
- Key Result: Bugs past target SLOs gitlab-org/gitlab-insights#118 (closed) => 100%`
-
Migrate charts in the Quality Dashboard which are targeted as EOL =>
50%
gitlab-org/gitlab-insights#116 (comment 185170975)
Retrospective
Good
- Charts are relatively easy for us to add to the Quality dashboard.
- Collaboration between @markglenfletcher and the data team seems to help close out the KPI working group. We have parity now except from the data in Dev.
- We added a few new charts that would help PMs and EMs prioritize and visualize the bug backlog.
- A few charts got migrated to Native GitLab Insights.
- We have more clarity on the direction of missed SLO detection.
Bad
- Data parity in periscope and quality dashboard caused uncertainty on where we should pin the location of our metrics. We can't shutdown the quality dashboard anytime soon. But we don't intend to add new development to it.
- Additional metrics being requested from dev metrics WG which is outside the scope of our KPIs
- Periscope does not have csv export, this adds more reliance on the quality dashboard.
- We didn't get to close out or polish Native GitLab insights.
Try
- Get all the high-level data 50,000 feet department view into Periscope.
- Ground truth for all groups still needs to be in the Quality dashboard (sustaining effort).
- Get all engineers in @gl-quality/eng-prod fluent in periscope so we can own escalation and speed to delivery, we already know what data to look for.
- Focus more on engineering productivity data; pipeline duration, review app, QA runtime, tests runtime.
- Get creative on building KPIs in periscope, it could be a chart that ingests a CSV to get something going while the CSV is generated outside of periscope via a generic ruby script.
- Fix Native GitLab Insights bugs.