Skip to content

Add 30s offset to sli_aggregations:* rules

Bob Van Landuyt requested to merge bvl/offset-recording-rule-registry into master

Add 30s offset to sli_aggregations: rules*

This offset will allow us to avoid a race condition where not all information in a numerator/denominator combination is not yet available when the rule is evaluated.

The 30s offset adds a delay, so that any metrics that come in late (with a timestamp in the past) would be part of the recording. This makes the numerator and denominator data a closer match most of the time, taking care of some deviations we've seen in ratio calculations used in alerts and error budgets for stage groups.

This is a stop-gap solution discussed in gitlab-com/gl-infra/scalability#2478 (closed)

For gitlab-com/gl-infra/scalability#2486 (closed)

Merge request reports