Establish a process to verify that our dashboards are accurate (pipeline durations, pipeline types, ...)
Context
In the part, we've had issues with our dashboards (1, 2) that heavily changed the data we were looking at.
We stumbled upon those issues by accident.
Goal
Have a process we do regularly to check that the pipeline durations we have is the reality that Engineers experience.
To verify
- Tableau dashboards (could be like “Snowflake has the same data“)
- Snowflake
- The monthly percentile/durations
- Very important, as KRs depend on this one.
- The pipeline types (for all pipelines and for predictive pipelines)
- The monthly percentile/durations
In particular, verify the following data:
- duration percentiles/average
- number of pipelines for a given period
- number of pipelines for each type
Technical notes
I've done such analysis in #422 (comment 1774900350) (under How did we verify locally?
). We need to formalize this approach, have it in a Runbook page, and extend it to verify pipeline types and predictive pipelines data.
Edited by David Dieulivol