Enable monitoring and Grafana BGP dashboard for MetalLB frr-k8s mode
This issue involves implementing monitoring capabilities and adapting MetalLB Grafana dashboard to visualize BGP metrics for MetalLB when running with the FRR-K8s mode.
Configure Prometheus integration to scrape BGP-related metrics from frr-k8s pods instead of MetalLB speakers.
Example of values to enable frr-k8s:
```yaml
metallb:
bgp_lbs:
l3_options:
bgp_peers:
ext-router1:
local_asn: 64513
peer_asn: 64513
peer_address: 172.20.219.241
advertised_pools:
- pool1
receive_routes:
mode: all
address_pools:
pool1:
addresses:
- 192.168.1.1-192.168.1.2
```
Tasks:
(A) This config in metallb helm frr-k8s values helps in exposing frr-k8s metrics:
```
metallb_helm_values:
frr-k8s:
prometheus:
namespace: cattle-monitoring-system
rbacPrometheus: true
serviceAccount: rancher-monitoring-prometheus
serviceMonitor:
enabled: true
rbacProxy:
repository: quay.io/brancz/kube-rbac-proxy
tag: v0.12.0
```
(B) For Grafana metallb-dashboard configmap needs to be adapted because of different metric names:
| Mode | BGP Metric Name |
|------|-----------------|
| Native BGP | `metallb_bgp_session_up` |
| frr-k8s | `frrk8s_bgp_session_up` |
In **frr-k8s mode**, the `frr-metrics` container exports metrics with the `frrk8s_bgp_*` prefix. The Grafana dashboard queries `metallb_bgp_*`.
(C) For Thanos, `sylva-thanos-rules-configmap` namespace contains existing MetalLB BGP alert rules in its `metallb.yml` key:
- `MetalLB-BGP_Session_Down` — fires when `metallb_bgp_session_up == 0`
- `MetalLB-BGP_All_Sessions_Down` — fires when all `metallb_bgp_session_up` are 0
Both query `metallb_bgp_session_up`, the alerts will never fire for any cluster running in frr-k8s mode. Patch the `metallb.yml` key in `sylva-thanos-rules-configmap` to extend each BGP alert expression with an `or` covering `frrk8s_bgp_session_up`.
(D) Adapt also how values are passed to sylva-capi-cluster for frr-k8s mode for first node, to omit serviceMonitor enablement similar to what is done for MetalLB: https://gitlab.com/sylva-projects/sylva-core/-/blob/1.6.18/charts/sylva-units/values.yaml?ref_type=tags#L10652
To avoid errors like:
```shell
wc1-vlado-cp-081544d333-c5fbd:~ # kubectl -n metallb-system logs helm-install-metallb-6875q
...
Error: INSTALLATION FAILED: unable to build kubernetes objects from release manifest: resource mapping not found for name: "metallb-frr-k8s-frr-k8s-monitor" namespace: "metallb-system" from "": no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
ensure CRDs are installed first
```
issue