Saturation metrics for HPA
Currently we do not have a view into the saturation of our HPA's in Kubernetes. We may need a complicated query to ensure that we are able to gather the appropriate data to build a view into saturation.
Get our HPA maxPods configuration via: kube_hpa_spec_max_replicas
Pods are owned by replicasets, replicasets are owned by deployments. We are able to get data from kube_replicaset_spec_replicas
to give us how many Pods are currently configured by that active replicaset. We then need to find that replicaset's owner via kube_replicaset_owner
using the owner_name
label.
Milestones
-
Construct a usable prometheus query -
Add it to our dashboards -
Add it to our alerting
Edited by John Skarbek