Grafana OIDC restart policy fails when rancher-monitoring-grafana deployment doesn't exist during secret update
It was observed in random CI runs that the mgmt-policy-report test is failing (see https://gitlab.com/sylva-projects/sylva-core/-/jobs/11501190022), which is being caused by the policy attempting to mutate the Deployment in cases where it did not exist / was not yet ready.
policyreport a31ab19f-2476-4915-a163-249b38cbefa7 in cattle-monitoring-system namespace was in FAIL/WARN/ERROR
scope:
apiVersion: v1
kind: Secret
name: grafana-oidc
namespace: cattle-monitoring-system
uid: a31ab19f-2476-4915-a163-249b38cbefa7
results:
- message: ': deployments.apps "rancher-monitoring-grafana" not found'
policy: cattle-monitoring-system/restart-deployment-when-oidc-secret-change
properties:
process: background scan
result: error
rule: update-oidc-secret
scored: true
source: kyverno
timestamp:
nanos: 0
seconds: 1758837770
We should add a precondition to the Kyverno policy, in order to wait for the deployment to be ready, before trying to mutate it.