Add Goldpinger cluster_health_total alert rule
The existing Goldpinger node_unhealthy alert fires on:
goldpinger_nodes_health_total{status="unhealthy"} > 0This only detects pod-to-pod connectivity failures between goldpinger instances. It does not fire when TCP (to cluster VIP port 6443) or HTTP (to goldpinger service) probes configured via TCP_TARGETS / HTTP_TARGETS fails on a node — even if that node reports goldpinger_cluster_health_total == 0.
This metric gives a better view on the whole cluster networking health since it checks core-dns functionality (when resolving the goldpinger service name), kube-proxy functionality on each node (for service reachability), cluster vip reachability from each node.
Add a new alert rule for both management and workload clusters to include also this check:
- alert: Goldpinger_Cluster_Health_Unhealthy
expr: goldpinger_cluster_health_total == 0
for: 5m
labels:
severity: 'critical'
sylva_alert: 'true'
type: 'network'
annotations:
description: 'Goldpinger reports overall cluster health failure on node "{{ $labels.goldpinger_instance }}". This indicates a pod-to-pod connectivity failure or failure to reach goldpinger service or cluster VIP on port 6443.'
runbook_url: ''
summary: 'Goldpinger overall cluster health is 0 on node {{ $labels.goldpinger_instance }}'