Alert notification not working for managed Prometheus installations

Summary

Alert notifications for manager Prometheus installations are broken since https://gitlab.com/gitlab-org/gitlab-ee/commit/3121445590e8eceac89972de258ed81562720ad0.

It seems that the essential code (note the presence of the bearer_token) got lost during a rebase (note the absence of the bearer_token) of security MR (internal dev server) unintentionally and undetected. So, the bearer_token never been exposed to Kubernetes installations.

Steps to reproduce

On GDK:

  1. Setup Prometheus
  2. Deploy to an environment
  3. Configure an alert which will be ultimately firing
  4. Wait until Prometheus and the Alertmanager has been updated
  5. Wait for alert notification mail to arrive
  • ➡ nothing happens
  1. Check Alertmanager's ConfigMap via kubectl get configmaps prometheus-alertmanager -n gitlab-managed-apps -o yaml
  • ➡ missing http_config and bearer_token within webhook_configs

Example Project

Run GDK locally

What is the current bug behavior?

The http_config.bearer_token within webhook_configs is missing

What is the expected correct behavior?

The http_config.bearer_token should be configured within webhook_configs.

Possible fixes

Bring back the lost code.

/cc @joshlambert @sengelhard @rpereira2

Edited by Peter Leitzen