Turn the get-hybrid monitoring config into a monitoring mixin
Monitoring mixins look like they could be becoming a standard way of sharing dashboards, rules and alerts. If we were to extract all of the jsonnet out of the runbooks to generate these files, we could publish our get-hybrid monitoring configuration.
With this, customers could use our dashboards and alerts to monitor their GitLab installation, and we might be able to use this more easily for Cells and dedicated.
If I understand everything correctly, a mixin exposes 3 fields in jsonnet:
├── grafanaDashboards::
├── prometheusAlerts::
│ ├── groups:
├── prometheusRules::
│ ├── groups:
├── _config::
This is already very close with what our get-hybrid
source looks like, the results are in generate.jsonnet
(import 'dashboards/dashboards.jsonnet')
+
(import 'rules/rules.jsonnet')
There each entry contains a yaml => content
mapping, but we could keep this in jsonnet, and leave the rendering up to mixtool. We would have to split up rules
into prometheusAlerts
and prometheusRules
. But I think this would be feasible
Our _config
object is already extracted out into: gitlab-metrics-config.libsonnet
, which we access all over the place. The way we work with this is by specifying the the gitlab-metrics-config
we want to use on the right when we pass in the library paths for jsonnet. This allows dedicated (or the get-hybrid
itself) to specify a different gitlab-metrics-config
for a tenant.