Route SLO alerts to pagerduty

gitlab-com/runbooks!1344 (merged) is routing latency SLO alerts to pagerduty as they seem to be very accurate in indicating real production issues.

We also should send other SLO alerts in https://gitlab.com/gitlab-com/runbooks/blob/master/rules/general-service-alerts.yml to pagerduty after we trimmed down the false alert rate (https://gitlab.com/gitlab-org/gitlab-ce/issues/66166):

  • error ratios alerts
  • saturation alerts
  • service availability alerts

Operation rate alerts probably shouldn't go to pagerduty, as they will manifest in higher latencies or error ratios when they affect production.

Edited by Henri Philipps