prometheus OOM
In my dev env (CAPO 3 CP nodes, 3 workers), prometheus containers get killed because out-of-memory:
(dmesg):
[ 2139.428812] oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=cri-containerd-ee2523e80ed899d2aab15888cb0c19d2660e36a76353b8448feebfad0d0c8647.scope,mems_allowed=0,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afcfe0b_751a_493e_932c_31abad059b3c.slice/cri-containerd-ee2523e80ed899d2aab15888cb0c19d2660e36a76353b8448feebfad0d0c8647.scope,task_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1afcfe0b_751a_493e_932c_31abad059b3c.slice/cri-containerd-ee2523e80ed899d2aab15888cb0c19d2660e36a76353b8448feebfad0d0c8647.scope,task=prometheus,pid=20504,uid=1000
[ 2139.428880] Memory cgroup out of memory: Killed process 20504 (prometheus) total-vm:3845168kB, anon-rss:3063592kB, file-rss:50204kB, shmem-rss:0kB, UID:1000 pgtables:6276kB oom_score_adj:954