Make more use of CEL preconditions in Kyverno ClusterPolicies
What does this MR do and why?
As informed in https://github.com/kyverno/kyverno/issues/10745#issuecomment-2264386929 and https://kyverno.io/docs/writing-policies/match-exclude/#configure-webhooks by not setting operations for the request, the policy defaults to all operations:
If for a resource no operations are set in match or exclude blocks, default operations are applied in the webhooks rules. Default operations for validating resources are CONNECT, CREATE, UPDATE, DELETE and for mutating resources are CREATE, UPDATE.
That is why, apart of adjusting to CEL syntax for ClusterPolicy/disallow-latest-and-main-tag & ClusterPolicy/disallow-default-namespace, we've added:
spec:
webhookConfiguration:
matchConditions:
- name: 'is-create-or-update'
expression: 'request.operation in ["CREATE", "UPDATE"]'otherwise, resources would be deleted (like when HelmRelease would be uninstalled, for example), errors like:
resource Job/cattle-monitoring-system/rancher-monitoring-crd-create was blocked due to the following policies
disallow-default-namespace:
validate-podcontroller-namespace: 'expression ''!(object.metadata.namespace == ''default'')''
resulted in error: no such key: metadata' would be seen.
For ClusterPolicy/pdb-minavailable-check, I didn't think the PDB can be queried with CEL in, per examples in https://kyverno.io/blog/2023/11/13/using-cel-expressions-in-kyverno-policies/#cel-variables-in-kyverno-policies, so only changed the precondition for the operation. Please do inform if you think that should be possible.
Related reference(s)
Closes #1501 (closed)
Test coverage
Manual tests results:
- ClusterPolicy/disallow-latest-and-main-tag
$ kubectl -n test-workload1 run network-testing --image praqma/network-multitool
Error from server: admission webhook "validate.kyverno.svc-fail-finegrained-disallow-latest-and-main-tag" denied the request:
resource Pod/test-workload1/network-testing was blocked due to the following policies
disallow-latest-and-main-tag:
require-image-tag: An image tag is required
$
$ kubectl -n test-workload1 run network-testing --image praqma/network-multitool:main
Error from server: admission webhook "validate.kyverno.svc-fail-finegrained-disallow-latest-and-main-tag" denied the request:
resource Pod/test-workload1/network-testing was blocked due to the following policies
disallow-latest-and-main-tag:
validate-image-tag: Using a mutable image tag e.g. 'latest' or 'main' is not allowed
$
$ kubectl -n test-workload1 run network-testing --image praqma/network-multitool:latest
Error from server: admission webhook "validate.kyverno.svc-fail-finegrained-disallow-latest-and-main-tag" denied the request:
resource Pod/test-workload1/network-testing was blocked due to the following policies
disallow-latest-and-main-tag:
validate-image-tag: Using a mutable image tag e.g. 'latest' or 'main' is not allowed
$- ClusterPolicy/disallow-default-namespace
$ kubectl -n default run network-testing --image praqma/network-multitool
Error from server: admission webhook "validate.kyverno.svc-fail-finegrained-disallow-default-namespace" denied the request:
resource Pod/default/network-testing was blocked due to the following policies
disallow-default-namespace:
validate-namespace: Using 'default' namespace is not allowed.
$Another way to test it is CEL playground, for example kustomize-units/kyverno-policies/generic/tag.yaml change in CEL playground here.
I found strange the logs for Kyverno admission-controller still present after this change, with a lot of entries from DaemonSet/calico-node and Deployment/calico-typha and Deployment/calico-kube-controllers, but this seems to be a different issue tracked by #1556 (closed).
kubectl -n kyverno logs deploy/kyverno-admission-controller --since 5m | grep disallow-default-namespace (click to expand)
$ kubectl -n kyverno logs deploy/kyverno-admission-controller --since 5m | grep disallow-default-namespace
Found 3 pods, using pod/kyverno-admission-controller-6b7f579cdf-5qz8f
Defaulted container "kyverno" out of: kyverno, kyverno-pre (init)
2024-08-28T09:47:18Z INFO setup.cluster-policy logging/controller.go:45 resource added {"type": "ClusterPolicy", "name": "disallow-default-namespace"}
2024-08-28T09:47:19Z INFO webhooks.resource.validate validation/validation.go:125 validation passed {"gvk": "apps/v1, Kind=DaemonSet", "gvr": {"group":"apps","version":"v1","resource":"daemonsets"}, "namespace": "calico-system", "name": "calico-node", "operation": "UPDATE", "uid": "a1e9c518-0528-427f-92f5-d952bd9ff06a", "user": {"username":"system:serviceaccount:tigera-operator:tigera-operator","uid":"f0208885-1c09-4bd3-b51a-2912743c4716","groups":["system:serviceaccounts","system:serviceaccounts:tigera-operator","system:authenticated"],"extra":{"authentication.kubernetes.io/pod-name":["tigera-operator-9c8fd8cc7-82xx7"],"authentication.kubernetes.io/pod-uid":["c32d2784-8731-4663-8b0a-87ddcc90b82a"]}}, "roles": [], "clusterroles": ["system:basic-user", "system:discovery", "system:public-info-viewer", "system:service-account-issuer-discovery", "tigera-operator"], "resource.gvk": "apps/v1, Kind=DaemonSet", "kind": "DaemonSet", "URLParams": "/disallow-default-namespace", "action": "validate", "resource": "calico-system/DaemonSet/calico-node", "operation": "UPDATE", "gvk": "apps/v1, Kind=DaemonSet", "policy": "disallow-default-namespace"}
2024-08-28T09:47:19Z INFO webhooks.resource.validate validation/validation.go:125 validation passed {"gvk": "apps/v1, Kind=Deployment", "gvr": {"group":"apps","version":"v1","resource":"deployments"}, "namespace": "calico-system", "name": "calico-kube-controllers", "operation": "UPDATE", "uid": "7e8751ea-a4f8-46f2-a9ea-ca1cf774bcfd", "user": {"username":"system:serviceaccount:tigera-operator:tigera-operator","uid":"f0208885-1c09-4bd3-b51a-2912743c4716","groups":["system:serviceaccounts","system:serviceaccounts:tigera-operator","system:authenticated"],"extra":{"authentication.kubernetes.io/pod-name":["tigera-operator-9c8fd8cc7-82xx7"],"authentication.kubernetes.io/pod-uid":["c32d2784-8731-4663-8b0a-87ddcc90b82a"]}}, "roles": [], "clusterroles": ["system:basic-user", "system:discovery", "system:public-info-viewer", "system:service-account-issuer-discovery", "tigera-operator"], "resource.gvk": "apps/v1, Kind=Deployment", "kind": "Deployment", "URLParams": "/disallow-default-namespace", "action": "validate", "resource": "calico-system/Deployment/calico-kube-controllers", "operation": "UPDATE", "gvk": "apps/v1, Kind=Deployment", "policy": "disallow-default-namespace"}
2024-08-28T09:47:20Z INFO webhooks.resource.validate validation/validation.go:125 validation passed {"gvk": "apps/v1, Kind=Deployment", "gvr": {"group":"apps","version":"v1","resource":"deployments"}, "namespace": "rke2-bootstrap-system", "name": "rke2-bootstrap-controller-manager", "operation": "UPDATE", "uid": "a6367829-01ff-4b00-be76-4bcf2daa6367", "user": {"username":"system:serviceaccount:flux-system:kustomize-controller","uid":"1c1a7ebf-b3e4-4d10-a113-38f030626269","groups":["system:serviceaccounts","system:serviceaccounts:flux-system","system:authenticated"],"extra":{"authentication.kubernetes.io/pod-name":["kustomize-controller-674d69cb6d-fmcvp"],"authentication.kubernetes.io/pod-uid":["5c47a4e1-c3ed-45ba-b4e9-e6c8645b34d2"]}}, "roles": [], "clusterroles": ["cluster-admin", "crd-controller", "system:basic-user", "system:discovery", "system:public-info-viewer", "system:service-account-issuer-discovery"], "resource.gvk": "apps/v1, Kind=Deployment", "kind": "Deployment", "URLParams": "/disallow-default-namespace", "action": "validate", "resource": "rke2-bootstrap-system/Deployment/rke2-bootstrap-controller-manager", "operation": "UPDATE", "gvk": "apps/v1, Kind=Deployment", "policy": "disallow-default-namespace"}
:
:
2024-08-28T09:50:28Z INFO webhooks.resource.validate validation/validation.go:125 validation passed {"gvk": "apps/v1, Kind=Deployment", "gvr": {"group":"apps","version":"v1","resource":"deployments"}, "namespace": "calico-system", "name": "calico-typha", "operation": "UPDATE", "uid": "1f1ae690-c6de-4449-8a85-cd151b0a014c", "user": {"username":"system:serviceaccount:tigera-operator:tigera-operator","uid":"f0208885-1c09-4bd3-b51a-2912743c4716","groups":["system:serviceaccounts","system:serviceaccounts:tigera-operator","system:authenticated"],"extra":{"authentication.kubernetes.io/pod-name":["tigera-operator-9c8fd8cc7-82xx7"],"authentication.kubernetes.io/pod-uid":["c32d2784-8731-4663-8b0a-87ddcc90b82a"]}}, "roles": [], "clusterroles": ["system:basic-user", "system:discovery", "system:public-info-viewer", "system:service-account-issuer-discovery", "tigera-operator"], "resource.gvk": "apps/v1, Kind=Deployment", "kind": "Deployment", "URLParams": "/disallow-default-namespace", "action": "validate", "resource": "calico-system/Deployment/calico-typha", "operation": "UPDATE", "gvk": "apps/v1, Kind=Deployment", "policy": "disallow-default-namespace"}
2024-08-28T09:50:28Z INFO webhooks.resource.validate validation/validation.go:125 validation passed {"gvk": "apps/v1, Kind=DaemonSet", "gvr": {"group":"apps","version":"v1","resource":"daemonsets"}, "namespace": "calico-system", "name": "calico-node", "operation": "UPDATE", "uid": "20fb4010-0116-46c0-adb6-dbf2d95b8ade", "user": {"username":"system:serviceaccount:tigera-operator:tigera-operator","uid":"f0208885-1c09-4bd3-b51a-2912743c4716","groups":["system:serviceaccounts","system:serviceaccounts:tigera-operator","system:authenticated"],"extra":{"authentication.kubernetes.io/pod-name":["tigera-operator-9c8fd8cc7-82xx7"],"authentication.kubernetes.io/pod-uid":["c32d2784-8731-4663-8b0a-87ddcc90b82a"]}}, "roles": [], "clusterroles": ["system:basic-user", "system:discovery", "system:public-info-viewer", "system:service-account-issuer-discovery", "tigera-operator"], "resource.gvk": "apps/v1, Kind=DaemonSet", "kind": "DaemonSet", "URLParams": "/disallow-default-namespace", "action": "validate", "resource": "calico-system/DaemonSet/calico-node", "operation": "UPDATE", "gvk": "apps/v1, Kind=DaemonSet", "policy": "disallow-default-namespace"}
$Edited by Bogdan-Adrian Burciu