Dedicate a partition for the local-Path-Provisioner

By default, when longhorn is disabled, on BM deployments with rke2, volumes are created by the local-path-provisioner, that is creating directories into /opt.

Using the new secured image, we have discovered that the logical volume partition for root has only 5 GB as size and also there is no dedicated /opt partition.

Looking on the os-images namespace we have seen below error which triggered our attention:

Image URL changed (new: oci://registry.gitlab.com/sylva-projects/sylva-elements/diskimage-builder/diskimage-builder-hardened:ubuntu-22.04-full, old: ). File ubuntu-22.04-full.qcow2 must be downloaded.
sh: write error: No space left on device

Checking the linux disk space we have seen that there is no space left on the /dev/mapper/vg-lv-root :

dev/mapper/vg-lv_root         4.6G  4.3G     0 100% /
/dev/mapper/vg-lv_home          22G   72K   22G   1% /home
/dev/mapper/vg-lv_tmp           22G   52K   22G   1% /tmp
/dev/mapper/vg-lv_var           22G  1.2G   20G   6% /var
/dev/mapper/vg-lv_kubelet       88G  166M   85G   1% /var/lib/kubelet
/dev/mapper/vg-lv_containerd   132G   27G  101G  21% /var/lib/rancher/rke2/agent/containerd
/dev/mapper/vg-lv_varlog        22G  147M   21G   1% /var/log
/dev/mapper/vg-lv_vartmp        22G   48K   22G   1% /var/tmp
/dev/mapper/vg-lv_etcd          44G  494M   42G   2% /var/lib/rancher/rke2/server/db
/dev/mapper/vg-lv_varlogaudit   22G  4.2M   22G   1% /var/log/audit

While checking on the failing container the mounted partitions, we have seen that the destination is mapped to /opt/local-path-provisioner.

       "destination": "/var/www/os-images/",
          "type": "bind",
          "source": "/opt/local-path-provisioner/pvc-baf9388c-e248-4d9c-98b6-2f512e15c03b_os-images_os-image-server-ubuntu-2204-full-qcow2",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },

Actually this is the location where all pv claims for local-path-provisioner will be created.

cat /var/lib/kubelet/pods/091ecb63-23b4-46f5-8d8b-234dd8a0a3e5/volumes/kubernetes.io~configmap/config-volume/config.json
{
  "nodePathMap": [
    {
      "node": "DEFAULT_PATH_FOR_NON_LISTED_NODES",
      "paths": [
        "/opt/local-path-provisioner"
      ]
    }
  ] 

This is the image which was used during this tests:

os_images:
  ubuntu-2204-full-qcow2:
    uri: "{{ .Values.sylva_base_oci_registry }}/sylva-elements/diskimage-builder/diskimage-builder-hardened:ubuntu-22.04-full"
    filename: ubuntu-22.04-full.qcow2
    checksum: 6ea1be09d24f09cff3c68356035b0f31d4237c188c75e7562631a02bd2405167
Edited by Mathieu Rohon