monitoring stack refactoring to use Prometheus community chart instead of Rancher charts

Summary

To make sure we are as close as possible to upstream versions of services such as Grafana, we should start exploring on what we would need to do to decouple from the cattle-monitoring-system chart for deploying Grafana and Prometheus.

One of the reasons for this would be the fact that we are constantly behind the upstream versions, for example, in main we currently deploy grafana 10.3.3 , the upstream version for grafana is 11.3, prometheus is at 2.50.1 as opposed to 2.55 (v3.0 is in rc).

Having the latest upstream versions would allow us to benefit of security and bugfixes without having to wait for a release provided by the cattle-monitoring charts.

related references

Details