Prep for Ops Section PI Review - 2020-08-06
📖 References
🎯 Intent
We are organizing for and regularly reviewing Performance Indicators (PIs) in order to enable a dialog between ourselves (all of R&D) about most effective use of our R&D efforts towards the most impactful improvements. As a result we should:
- focus on singular Performance Indicators
- focus on the art of the possible, what we can measure and get react to quickly
- focus on business metrics, particularly SMAU
- continue beyond measurement and review of our Performance Indicators to building a Growth Model, a Growth funnel and supporting metrics and segmentation
As a communication tool, Performance Indicators are only as useful as the range of the audience. As a result we should:
- store PI metrics, funnels, growth models, status and next steps handbook first (avoid content only living in slides)
- expose our product groups to PI metrics regularly
- communicate transparently about adjustments to PIs within our groups
- reference PIs in planning issues and prioritization discussions
✅ Tasks
⭕ Opening Tasks
-
Set a due date to this issue as 2 days prior to the scheduled review - @kencjohnston -
Update the reference to the this Prep issue in the first slide of the presentation - @kencjohnston -
Add a restrospective thread to this issue - @kencjohnston -
Update Agenda - @kencjohnston
✔ Other Tasks
-
Iterate on Over-estimate Non-Viable SMAU - @kencjohnston -
Move intent content to Ops Section PI page - @kencjohnston - www-gitlab-com!58678 (merged) -
Add sequence of PI adoption (define, instrument, set goal, define growth model, understand funnel) to handbok and remove from this issue template - @kencjohnston - www-gitlab-com!57078 (merged)
⭐ Performance Indicator YAML Updates
Each product stage or group should update the Performance Indicators YAML file with:
- Update your
health:
andinstrumentation:
levels - Review your
definition
to include both what specifically is being measured and the rationale for why you chose that measurement - Update any
urls:
to reference stage/group direction or handbook pages for the growth model and funnel - Update any
urls:
to reference additional Sisense dashboards for more details - Update
health:
reasons:
withInsight -
prefixed comments about insights gained from the last month of data - Update
health:
reasons:
withImprovement -
prefixed comments about planned improvements to reach your goals over the next month - If there have been recent updates to the definition, growth model, funnel highlight those and include those as
reasons:
underhealth:
- If you don't have a defined metric, rationale, growth model or funnel in the handbook reference issues for creating those and provide status on blockers in the
reasons:
underinstrumentation:
.
Group Updates
-
Update Section Wide - @kencjohnston - MR -
Update Verify:CI - @thaoyeager - www-gitlab-com!58147 (merged) -
Update Verify:Runner - @DarrenEastman - www-gitlab-com!58147 (merged) -
Udpate Verify:Testing - @jheimbuck_gl - www-gitlab-com!57012 (merged) -
Update Package - @trizzi - www-gitlab-com!56887 (merged) -
Update Release:Release Management - @jmeshell - www-gitlab-com!56875 (merged) -
Update Release:Progressive Delivery - @ogolowinski - www-gitlab-com!56875 (merged) -
Update Configure - @nagyv-gitlab - www-gitlab-com!58082 (merged) -
Update Monitor:APM - @dhershkovitch - www-gitlab-com!58068 (merged) -
Update Monitor:Health - @sarahwaldner - www-gitlab-com!56866 (merged) and www-gitlab-com!58486 (merged)
2️⃣ Second Round Updates
We missed the mark on insights and improvements in the first round of updates.
Please focus on these specific instructions.
- Update
health:
reasons:
withInsight -
prefixed comments about insights gained from the last month of data - Update
health:
reasons:
withImprovement -
prefixed comments about planned improvements to reach your goals over the next month - If there have been recent updates to the definition, growth model, funnel highlight those and include those as
reasons:
underhealth:
- If you don't have a defined metric, rationale, growth model or funnel in the handbook reference issues for creating those and provide status on blockers in the
reasons:
underinstrumentation:
.
I'd really like to see Insight -
s for any instrumented metrics, Improvement -
s for all metrics, and issue references for all next steps. If you haven't instrumented the metric reference an issue for doing so, if you don't have a growth model have an issue for creating one.
-
Update Verify:CI - @thaoyeager - www-gitlab-com!58790 (merged) -
Update Verify:Runner - @DarrenEastman - www-gitlab-com!58790 (merged) -
Update Verify:Testing - @jheimbuck_gl - www-gitlab-com!58761 (merged) -
Update Release:Release Management - @jmeshell - www-gitlab-com!58469 (merged) -
Update Release:Progressive Delivery - @ogolowinski - www-gitlab-com!58450 (merged) and www-gitlab-com!58728 (merged) and www-gitlab-com!58733 (merged) -
Update Configure - @nagyv-gitlab - www-gitlab-com!58736 (merged) -
Update Monitor:APM - @dhershkovitch - www-gitlab-com!58744 (merged) -
Update Monitor:Health - @sarahwaldner - www-gitlab-com!58738 (merged)
❌ Closing Tasks
-
Share the PI Page in the #ops-section and #product Slack channels 24 hours in advance of the review - @kencjohnston -
Make adjustments to the template based on the retrospective thread - @kencjohnston