Add Goldpinger cluster_health_total alert rule

The existing Goldpinger node_unhealthy alert fires on:

goldpinger_nodes_health_total{status="unhealthy"} > 0

This only detects pod-to-pod connectivity failures between goldpinger instances. It does not fire when TCP (to cluster VIP port 6443) or HTTP (to goldpinger service) probes configured via TCP_TARGETS / HTTP_TARGETS fails on a node — even if that node reports goldpinger_cluster_health_total == 0.

image

This metric gives a better view on the whole cluster networking health since it checks core-dns functionality (when resolving the goldpinger service name), kube-proxy functionality on each node (for service reachability), cluster vip reachability from each node.

Add a new alert rule for both management and workload clusters to include also this check:

  - alert: Goldpinger_Cluster_Health_Unhealthy
    expr: goldpinger_cluster_health_total == 0
    for: 5m
    labels:
      severity: 'critical'
      sylva_alert: 'true'
      type: 'network'
    annotations:
      description: 'Goldpinger reports overall cluster health failure on node "{{ $labels.goldpinger_instance }}". This indicates a pod-to-pod connectivity failure or failure to reach goldpinger service or cluster VIP on port 6443.'
      runbook_url: ''
      summary: 'Goldpinger overall cluster health is 0 on node {{ $labels.goldpinger_instance }}'