Persist alerts coming from manual Promtheus installations
Summary
With https://gitlab.com/gitlab-org/gitlab-ee/issues/7756 we persist firing
For manual Prometheus installations, however, we don't persist alerts as we are missing labels/gitlab_alert_id
in the payload.
Steps to reproduce
- Enable manual configuration for Prometheus
- Configure Alertmanager accordingly to send notifications to GitLab
- Let Alertmanager fire
🔥 some alerts - No alerts are persisted in the database
What is the current bug behavior?
We do not track alerts coming from manual Prometheus installations.
What is the expected correct behavior?
We should persist alerts coming from manual Prometheus installations.
Possible fixes
Tweak PrometheusAlertEvent
:
- Make
prometheus_alert
optional - Add optional fields like
title
,description
andquery
- Deal with missing
gitlab_alert_id
and persist alerts to database - ~"technical debt" Pass a list of
Alerting::Alert
s toCreateEventsService
/cc @joshlambert