Set stabilizationWindowSeconds for HPA to a value longer than the 5 minute default

In production#3419 (closed) and production#3420 (closed) that we are seeing 5xx errors around scaling events. In both cases there was a fast scaling event due to a burst of traffic (in the case of websockets).

Should we consider setting stabilizationWindowSeconds?

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-cooldown-delay

I'm not clear if this is something we can set in Kubernetes 1.17, based on https://stackoverflow.com/questions/60045564/kubernetes-unknown-field-behavior it might be supported?

Status 2022-07-08

HPA scale-down policies have been adjusted on staging and production.

#1510 (comment 1030192200)

Edited by Amy Phillips