Add alarming for errors in ELK
Currently when there's an error in ELK, as long as nobody reports an issue, it goes unnoticed. It would be nice to have an alarming on errors to allow us to catch issues proactively and fix them before users report them.
Consider connecting to Mattermost or send an email if NOT possible. See https://www.elastic.co/de/what-is/kibana-alerting for more details as well.
Some possible approaches:
- Kibana's built-in alerting
- needs some setup, like TLS
- Has connectors for a webhook (which could be tried to use it with Mattermost) or alternatively email
-
Alerting Plugin by Yelp
- also requires some setup
- has connectors for email, but support for webhook / mattermost needs to be evaluated
Has to be implemented before #233 (closed). Relates to https://git.dcc.sib.swiss/biwg/elk-dev/-/issues/1.
Edited by François Martin