Add experimental recording rules with offset

Add experimental recording rules with offset

This adds a version of the recording rules with a 1m offset as an experiment to see if we can work around the jitter we're seeing in apdexes. The 1m offset here is rather unsatisfying because it adds another 1m of delay in recording rules, but this one 1m would at least be controlled rather than stacked up from recording rules.

More info in gitlab-com/gl-infra/scalability#2445 (comment 1503906053)

use abort for partial responses in experiment

This uses partial_response_strategy: abort for the experimental rules. I don't think this will make a lot of difference, because my understanding is that the partial responses strategy only affects the recordings when an entire thanos-sidecar fails to respond, which I don't think was the case in gitlab-com/gl-infra/scalability#2445 (comment 1503294764)

But I do think we should try to get to a place where our default is partial_response_strategy: abort rather than warn if we move to a world where all the recording happens in thanos. This is why we separate recordings per environment now.