Skip to content

Draft: Experimental aggregation sets for SLIs in thanos

Bob Van Landuyt requested to merge bvl-thanos-recordings into master

Draft: Experimental aggregation sets for SLIs in thanos

This adds 2 new aggregation sets recorded in thanos:

experimentalComponentSLIs: a replacement for the component slis that now get collected in Prometheus first and then in Thanos.

experimentalFeatureCategorySLIs: The same but for feature categories.

The metrics for these recording rules are all prefixed with experiment:.

The ratios are calculated from the previously recorded rates, similar to how we currently do it for Dedicated. This is still up for discussion, we could also use source metrics directly.

The following needs to be answered before we continue:

  1. Do we want to use source metrics for the ratios, or do we allow chaining to self?
  2. How can we run this on separate rulers so we don't make matters worse for other recordings.
  3. There's known bugs in this, but they currently don't affect recordings, see the comments.

Related issues:

Merge request reports