Add Mimir HPA support for components that are compatible

The mimir-distributed helm chart currently doesn't have any support for HPA.

We should look at adding this support into the chart (perferably with Keda), or in the short term we can bundle in the raw manifests with the helmfiles deployment.

The non stateful components we can confidently run HPA on for starters:

  • Querier
  • Query Frontend
  • Distributors
  • Rulers

See https://grafana.com/docs/mimir/latest/set-up/jsonnet/configure-autoscaling/ for reference on the jsonnet deployment which supports autoscaling.

Edited by Nick Duff