feat: Send Amp cluster name as the value for the env label
What
feat: Send Amp cluster name as the value for the env label
Related to https://gitlab.com/gitlab-com/gl-infra/delivery/-/work_items/21219
The env label is used by Alert Manager to route alerts to the right contact point. We are using a
combination of team and env to get alerts for cellsdev. So, we should set the appropriate
env label in the metric, allowing us to have one alert rule for both the Amp clusters.
We are using this routing rule in the Alert manager configuration: alertmanager.jsonnet This configuration uses the list of environments defined in the teams.yml file to decide where to route alerts. For instance, this block tells Alert Manager to send alerts for the cellsdev env to the g_delivery_alerts Slack channel.
(In the absence of this label, we will have to duplicate the rule for the two Amp clusters,
cellsdev and cellsprod)
Why
By specifying the list of environments that our team wants to get alerts for, we can receive alerts in Slack without changing the alert manager configuration. For this to work properly, all the metrics that we use in alerts must have the env label properly populated.
Links to relevant issues
https://gitlab.com/gitlab-com/gl-infra/delivery/-/work_items/21219