Skip to content

fix: simplify alert manager configuration to only require receivers

Keith Grootboom requested to merge keith/setup-default-prometheus-alerts into main

Description

Fixes the default Alert Manager config to make it simpler to set up a receiver and disables notifications for info alerts.

Without these, it's possible to flood an inbox with nothing really being wrong in the cluster.

Supporting information

https://tasks.opencraft.com/browse/SE-5475

Testing instructions

  • The changes have been deployed to the grove-stage-digitalocean repo. They can be viewed by going to the alert manager page https://alert-manager.staging.do.opencraft.hosting/#/status.
  • Emails have been sent to the ops@ mailing list. Verify that they have been sent from grove-stage-digitalocean--- via Ops.

Checklist

If any of the items below is not applicable, do not remove them, but put a check in it.

  • All providers include the new feature/change
  • All affected providers can provision new clusters
  • Unit tests are added/updated
  • Documentation is added/updated
  • The TOOLS_CONTAINER_IMAGE_VERSION in ci_vars.yml is updated
  • The grove-template repository is updated

Additional context

Once approved, I'll add the settings to the CI/CD. The settings differ from what would be production settings in that the repeat_interval is 8h instead of 1m.

Edited by Keith Grootboom

Merge request reports