Add AntiAffinity rules for the Kyverno pods (!7831) · Merge requests · Sylva-projects / sylva-core

What does this MR do and why?

This MR sets podAntiAffinity rules for the Kyverno pods, in order to schedule them on different nodes. The goal is to prevent all Kyverno pods from being placed on the same node, reducing the risk of complete disruption if that node becomes unavailable.

Closes #4016 (closed)

Test coverage

I tested it on my dev environment, on both management and workload clusters.

After the changes, this is the result on management cluster: (I edited the deployment replicas in order to check how the pods are scheduled on my 4 nodes)

kyverno-admission-controller-64945849d-82pns     1/1     Running     0          46h   100.72.177.170   teodora-cluster-cp-8f8390bc99-tt5vz   <none>           <none>
kyverno-admission-controller-64945849d-grfnc     1/1     Running     0          46h   100.72.39.58     teodora-cluster-md0-vfc5t-stmrc       <none>           <none>
kyverno-admission-controller-64945849d-pkh5f     1/1     Running     0          78s   100.72.100.185   teodora-cluster-cp-8f8390bc99-r6f9v   <none>           <none>
kyverno-admission-controller-64945849d-t9fj6     1/1     Running     0          46h   100.72.177.171   teodora-cluster-cp-8f8390bc99-tt5vz   <none>           <none>
kyverno-background-controller-5d48bf9f45-cnjg4   1/1     Running     0          45h   100.72.39.63     teodora-cluster-md0-vfc5t-stmrc       <none>           <none>
kyverno-background-controller-5d48bf9f45-l9j6c   1/1     Running     0          45h   100.72.100.167   teodora-cluster-cp-8f8390bc99-r6f9v   <none>           <none>
kyverno-background-controller-5d48bf9f45-rdshg   1/1     Running     0          45h   100.72.177.173   teodora-cluster-cp-8f8390bc99-tt5vz   <none>           <none>
kyverno-background-controller-5d48bf9f45-zlq66   1/1     Running     0          62s   100.72.136.163   teodora-cluster-cp-8f8390bc99-kdlzp   <none>           <none>
kyverno-cleanup-controller-b66f784bf-6mk27       1/1     Running     0          45h   100.72.39.6      teodora-cluster-md0-vfc5t-stmrc       <none>           <none>
kyverno-cleanup-controller-b66f784bf-6p2dl       1/1     Running     0          45h   100.72.177.176   teodora-cluster-cp-8f8390bc99-tt5vz   <none>           <none>
kyverno-cleanup-controller-b66f784bf-j4rhh       1/1     Running     0          45h   100.72.100.168   teodora-cluster-cp-8f8390bc99-r6f9v   <none>           <none>
kyverno-cleanup-controller-b66f784bf-m9f8j       1/1     Running     0          50s   100.72.136.164   teodora-cluster-cp-8f8390bc99-kdlzp   <none>           <none>
kyverno-reports-controller-bb675fbc7-dkw2v       1/1     Running     0          45h   100.72.136.153   teodora-cluster-cp-8f8390bc99-kdlzp   <none>           <none>
kyverno-reports-controller-bb675fbc7-kjfwk       1/1     Running     0          19s   100.72.100.187   teodora-cluster-cp-8f8390bc99-r6f9v   <none>           <none>
kyverno-reports-controller-bb675fbc7-sq66q       1/1     Running     0          45h   100.72.136.152   teodora-cluster-cp-8f8390bc99-kdlzp   <none>           <none>
kyverno-reports-controller-bb675fbc7-z2c68       1/1     Running     0          45h   100.72.177.175   teodora-cluster-cp-8f8390bc99-tt5vz   <none>           <none>

This is the result on workload cluster: (I edited the deployment replicas in order to check how the pods are scheduled on my 3 nodes)

kyverno-admission-controller-64945849d-6k8jq     1/1     Running   0          59s     100.72.23.8    wkld-teodora-md0-lvjsz-sqdt5       <none>           <none>
kyverno-admission-controller-64945849d-gsxpx     1/1     Running   0          3h57m   100.72.42.88   wkld-teodora-cp-69b7f77e9c-xt69m   <none>           <none>
kyverno-admission-controller-64945849d-rjm8z     1/1     Running   0          3h57m   100.72.42.90   wkld-teodora-cp-69b7f77e9c-xt69m   <none>           <none>
kyverno-background-controller-5d48bf9f45-8jqf8   1/1     Running   0          3h57m   100.72.42.89   wkld-teodora-cp-69b7f77e9c-xt69m   <none>           <none>
kyverno-background-controller-5d48bf9f45-dbbzj   1/1     Running   0          35s     100.72.76.7    wkld-teodora-md0-lvjsz-qmjh5       <none>           <none>
kyverno-background-controller-5d48bf9f45-nm26p   1/1     Running   0          35s     100.72.23.9    wkld-teodora-md0-lvjsz-sqdt5       <none>           <none>
kyverno-cleanup-controller-b66f784bf-brnnw       1/1     Running   0          19s     100.72.23.10   wkld-teodora-md0-lvjsz-sqdt5       <none>           <none>
kyverno-cleanup-controller-b66f784bf-j9cjt       1/1     Running   0          19s     100.72.76.8    wkld-teodora-md0-lvjsz-qmjh5       <none>           <none>
kyverno-cleanup-controller-b66f784bf-zzlwk       1/1     Running   0          3h57m   100.72.42.91   wkld-teodora-cp-69b7f77e9c-xt69m   <none>           <none>
kyverno-reports-controller-bb675fbc7-c4bsx       1/1     Running   0          5s      100.72.76.9    wkld-teodora-md0-lvjsz-qmjh5       <none>           <none>
kyverno-reports-controller-bb675fbc7-hd5f8       1/1     Running   0          5s      100.72.23.11   wkld-teodora-md0-lvjsz-sqdt5       <none>           <none>
kyverno-reports-controller-bb675fbc7-n9vpw       1/1     Running   0          3h57m   100.72.42.87   wkld-teodora-cp-69b7f77e9c-xt69m   <none>           <none>

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon	Meaning	Available values
☁️	Infra Provider	`capd`, `capo`, `capm3`
🚀	Bootstrap Provider	`kubeadm` (alias `kadm`), `rke2`, `okd`, `ck8s`
🐧	Node OS	`ubuntu`, `suse`, `na`, `leapmicro`
🛠️	Deployment Options	Deployment option list and description
🎬	Pipeline Scenarios	Available scenario list and description
🟢	Enabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by `mgmt:` or `wkld:` to be applied only to a specific cluster type
🔴	Disabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by `mgmt:` or `wkld:` to be applied only to a specific cluster type
🏗️	Target platform	Can be used to select specific deployment environment Available platform list and description
⚡	Pipeline control	`autorun`, `manual` or `blocking`. Can be used to override global config and start a deployment pipeline the required way

Global config for deployment pipelines

autorun pipelines
allow failure on pipelines
record sylvactl events

Notes:

Enabling autorun will make deployment pipelines to be run automatically without human interaction
Disabling allow failure will make deployment pipelines mandatory for pipeline success.
if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.

Add AntiAffinity rules for the Kyverno pods

What does this MR do and why?

Related reference(s)

Test coverage

CI configuration

Global config for deployment pipelines

Merge request reports