Kustomize unit for kata containers
What does this MR do and why?
This is the first MR to introduce kata containers in sylva. I have started with one infra provider that is capo, with this I would like to open up floor for feedback and discussions.
I have created a new kustomize unit in sylva-core by the name kata-deploy. The unit is applicable to management cluster for capo infra provider and both the bootstrap providers ( kubeadm and rke2 ), default state of the unit is disabled.
For other infra provider I will test and introduce at later stage. I have also introduced kyverno policy to enforce kata containers runtime based on "untrusted" label.
This feature needs to have hardware virtualization enabled on the underlying nodes. In the testing part, this setting seems to be enabled by default on the CAPO. For now, there is no validation implemented to check if it is enabled or not. In future, if required validation can be added.
Related reference(s)
Test coverage
Manual Steps -
- Check for kata-deploy daemonsets in kube-system namespace
- Check for runtimeclass kata-qemu.
- Check for kyverno policy by the name
kata-containers-on-untrusted-pod - Add a label
untrusted: "true"to pod, the runtimeclassname should be added as kata-qemu.
Output
Created two similar pods with a difference in label.
Basic Pod
apiVersion: v1
kind: Pod
metadata:
name: ubuntu
spec:
containers:
- name: ubuntu
image: ubuntu:20.04
command: ['sh', '-c', 'sleep 3600']
---
kubectl --kubeconfig management-cluster-kubeconfig describe po ubuntu
Name: ubuntu
Namespace: default
Priority: 0
Service Account: default
Pod with kata containers
apiVersion: v1
kind: Pod
metadata:
name: ubuntu-kata
labels:
untrusted: "true"
spec:
containers:
- name: ubuntu
image: ubuntu:20.04
command: ['sh', '-c', 'sleep 3600']
---
kubectl --kubeconfig management-cluster-kubeconfig describe po ubuntu-kata
Name: ubuntu-kata
Namespace: default
Priority: 0
Runtime Class Name: kata-qemu
Service Account: default
Kernel version output inside Pods-
