Add `safe-to-evict` annotation to stateless pods
From https://gitlab.com/charts/helm.gitlab.io/issues/313#note_66204875
@twk3 : I'm disabling the autoscaling for now, as it can't scale down when pods have local-storage (Most of our pods have some form of temporary local-storage on the node, and we would have to start adding the safe to evict annotation to them before we can auto-scale) At some point we may want to get that annotation added to our pods, then we can re-try this.
@WarheadsSE : Hmm, the ephemeral storage used for
initContainer
writingxxx-config
is blocking scaling/eviction? Well that is a downside. I went looking, but didn't find any flags/annotations that are applicable to set for this problem. Have you found any?
@twk3 : yeah, it's this
safe-to-evict
annotation. https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#what-types-of-pods-can-prevent-ca-from-removing-a-node
Add cluster-autoscaler.kubernetes.io/safe-to-evict: "true"
annotation to pods which are stateless, and those less sensitive to disruption (have proper graceful shutdown that does not interrupt other services)