Calculate Apdex score based on deployment metrics
As part of &461 (closed), we need to set a target to measure the time it takes for a deployment to rollout to production. We have two deployment metrics to capture deployment data:
- An histogram that keeps track of the pipeline duration in seconds, and
- A counter to keep track of the number of successful deployments.
Both metrics are pushed to the Prometheus PushGateway so they're available to Grafana (there's already a test dashboard with fake data in the grafana playground).
Proposal
Use the deployments metrics to implement an Apdex score in Grafana, the apdex score should display the % of pipelines completed in a specific time within our target bucket. To start with, we can use:
-
5.5h
as a target bucket. This bucket is defined in the duration histogram metric. - 95% should be used as a measure of satisfaction
- 1 month as a specific time
- e.g. 95% of our deployments within 1 month should last less than 5.5h
These values can be adjusted once we have more data. Final result of this issue should be similar to Apdex Score on this sketch. This issue should be just for implementing the Apdex Score, a later iteration will cover building the Deployment SLO Apdex graph as that graph requires the Apdex Score to be in place.
Implementation details
We have a Deployment SLO test dashboard that is fed manually by clicking a scheduled pipeline.
The Appdex score should be added to this dashboard.