Generate alert manager config from runbook pipeline

Currently, we have three different source of truth for our alert manager routing configuration.

These are:

  1. https://gitlab.com/gitlab-com/runbooks/-/blob/master/alertmanager/alertmanager.yml.erb
  2. https://gitlab.com/gitlab-cookbooks/gitlab-alertmanager/blob/master/templates/default/alertmanager.yml.erb
  3. An encrypted file, manually stored in GCS, and not regularly updated. See https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/monitoring/-/blob/master/bin/k-ctl#L57 for details

The first two are now relatively in check with one another, but effort needs to be made to unify all three routing configs.

Proposal

Add a pipeline to the runbooks repo to generate the config and store it encrypted in GCS.

Then adapt the chef repo to use this GCS file, rather than the ERB directly.

(Placeholder, more to follow)


Related:

cc @ggillies @skarbek @jarv @marin @craigf

Edited by Andrew Newdigate