disk pressure on Ubuntu nodes because missing mount points
In my dev environment I have both OpenSuse-hardened and Ubuntu-plain nodes.
I've noticed the Ubuntu nodes frequently being under disk pressure.
This is due to not having a lot of space in /var:
df -h |grep ^/ |grep -v dev/sd
/dev/mapper/vg-lv_root 4.6G 1.8G 2.5G 43% /
/dev/mapper/vg-lv_home 4.0G 52K 3.8G 1% /home
/dev/mapper/vg-lv_opt 1.8G 1.3G 413M 77% /opt
>> /dev/mapper/vg-lv_var 4.9G 3.8G 891M 81% /var <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
/dev/mapper/vg-lv_tmp 2.0G 32K 1.9G 1% /tmp
/dev/mapper/vg-lv_varlog 4.9G 24M 4.7G 1% /var/log
/dev/mapper/vg-lv_vartmp 919M 32K 856M 1% /var/tmp
The problem/inconsistency is that the overall disk is large, but it has most space on pre-existing volumes which are not mounted:
# lsblk -T -o NAME,SIZE
NAME SIZE
sda 100G
├─sda1 4M
├─sda2 250M
└─sda3 99.8G
├─vg-lv_root 4.7G
├─vg-lv_tmp 2G
├─vg-lv_var 5G
├─vg-lv_vartmp 2G
├─vg-lv_varlog 5G
├─vg-lv_varlogaudit 2G
^^ not used
├─vg-lv_home 4G
├─vg-lv_etcd 10G
^^ not used
├─vg-lv_containerd 29.9G
^^ not used
├─vg-lv_kubelet 29.9G
^^ not used
└─vg-lv_opt 1.9G
^^ not used
Comparatively, OpenSuse nodes are fine, because other volumes (larger ones) are mounted in some /var sub-directories:
# df -h |grep ^/ | grep -v dev/sd
/dev/mapper/vg-lv_root 4.6G 1.6G 2.7G 38% /
/dev/mapper/vg-lv_home 4.0G 64K 3.8G 1% /home
/dev/mapper/vg-lv_opt 1.8G 1.3G 413M 77% /opt
/dev/mapper/vg-lv_tmp 2.0G 40K 1.9G 1% /tmp
/dev/mapper/vg-lv_var 4.9G 1.4G 3.3G 30% /var
>> /dev/mapper/vg-lv_kubelet 30G 73M 29G 1% /var/lib/kubelet
>> /dev/mapper/vg-lv_containerd 30G 14G 15G 49% /var/lib/rancher/rke2/agent/containerd
>> /dev/mapper/vg-lv_etcd 9.9G 24K 9.4G 1% /var/lib/rancher/rke2/server/db
/dev/mapper/vg-lv_varlog 4.9G 53M 4.7G 2% /var/log
/dev/mapper/vg-lv_vartmp 2.0G 40K 1.9G 1% /var/tmp
>> /dev/mapper/vg-lv_varlogaudit 2.0G 400K 1.9G 1% /var/log/audit