feat(metrics-catalog): add requestRateDropAlertConfig for SLI request rate drops

The traffic cessation alerts fire when an SLI's request rate is exactly zero or the series is absent. They do not fire on a partial drop, such as merge request pipelines stopping while push pipelines continue (INC-11596). An SLI with only a request rate also has no apdex or error ratio that burns error budget.

The service ops rate anomaly detection does not cover this either. Its threshold is three standard deviations of the rate over a week. For a strongly diurnal signal that spread is wide: a 98% drop in merge_request_event pipelines replayed over that incident reaches -2.9 sigma. For ci-orchestration those rules evaluate against an empty vector, because every SLI has serviceAggregation: false.

This adds an opt-in SLI option, requestRateDropAlertConfig. It compares the request rate with its median at the same time of week over the previous three weeks and alerts when the ratio is below a threshold. The rate can be split by one significant label with a threshold per value. Absent series count as zero and a complete stop fires the same alert. The configuration reference is in the metrics catalog docs.

No SLI opts in here and the generated output is unchanged. !11233 (merged) is the first adopter.

Related: https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29301

Merge request reports

Loading
Loading