Persist Prometheus alert status
In order to have a more complete data model within GitLab about the health of an environment, we should persist the status of Prometheus alerts.
- We are already notified of an alert being triggered
- We are working on adding support for notifications of resolved alerts: https://gitlab.com/gitlab-org/gitlab-ee/issues/7495
As part of these handlers, we should start to keep track of the state of each alert in the GitLab database. This will allow us to quickly and easily present the alert status to users, for example on our Ops Homepage: https://gitlab.com/gitlab-org/gitlab-ee/issues/5781