refactor: update the way we match kubernetes resources with services
- Part of #78 (closed)
- Blocker on !4325 (merged)
kubeLabelSelector allows services in the metrics-catalog to define a set of kubernetes resources that belong to the service.
kubeConfig: {
labelSelectors: kubeLabelSelectors(
ingressSelector={ namespace: "monitoring" },
)
},In the example above, the service will include all ingresses in the namespace "monitoring". These will be included for monitoring, alerting and charting purposes.
This functionality relies on kube_state_metrics, which will create kube_<resource>_label metrics for each resource. The metrics have label_<kube_label> labels on them. This allows us to match Kubernetes label to services, using PromQL selector syntax.
Ideally, these selectors will be standardized on type, shard, etc but there are still a lot of exceptions, and this appproach allows us to flexibly include and exclude resources.
In developing this change, I've found a lot of inconsistencies in our labelling for gitlab.com. I've documented these issues in https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/15208.
In implementing this, I also discovered that we can delete rules-jsonnet/kube-node-pool-additional-labels.jsonnet which is an extremely fragile and complicated file. As a nice side-effect of this change, we can remove that entirely.