fix Kyverno policies used for firewalling for capm3
What does this MR do and why?
Fixes firewall-kyverno-capm3's Kyverno generate rules, which fail (or silently stay empty) on a
fresh capm3 deploy. They read Metal3 IPPool.status.indexes, but that field is written
asynchronously — and for the rules' current trigger, it can never be populated in time. That breaks
the firewall-common unit (management and every workload cluster), whose Kustomization
non-optionally substitutes from a Kyverno-generated ConfigMap:
post build failed for 'KubeControllersConfiguration.v1.crd.projectcalico.org/default':
substitute from 'ConfigMap/management-cluster-addresses' error: configmaps "management-cluster-addresses" not foundRelated reference(s)
Root cause and fix
Each rule matches only the base IPPool resource. But status.indexes — the addresses
metal3-ipam allocates — is written via a separate status-subresource PATCH, sent after the
IPPool object is created. A webhook matched only on the base resource never sees that second
request, so the rule fires exactly once, on CREATE, before any addresses exist — and nothing tells
it to look again once the real data lands. Not a timing issue: a hole in the trigger model.
Fix: also match ipam.metal3.io/*/IPPool/status. The status write becomes a trigger in its own
right, so whenever the rule runs, status.indexes reflects what was just written. The data source
itself doesn't change — it was always correct, it just needed a trigger that could see it update.
Match IPPool/status alone it breaks generateExisting: true (the backfill for a
source that already exists — e.g. policy redeploy, or metal3-ipam reconciling before the policy is
applied). Verified live: with only IPPool/status matched, the background controller never even
attempts a backfill. generateExisting needs to list matching objects first, and Kubernetes has
no LIST endpoint for a subresource — only GET <name>/status on an object it already knows about.
So IPPool covers backfill, IPPool/status covers the real-time update; neither alone is enough.
Before / after (same fresh-deploy trace, management cluster, times UTC):
IPPool CREATE |
status.indexes PATCH (metal3-ipam resumes after CAPI pause) |
|
|---|---|---|
Before (IPPool only) |
12:20:08Z — webhook fires, status.indexes is nil |
12:20:38Z — invisible to the webhook, no further attempt |
After (IPPool + IPPool/status) |
12:20:08Z — webhook fires, still empty (unavoidable) |
12:20:38Z — also matches → generate re-fires → correct output |
Changes
The 3 changes below apply consistently across the 5 kustomize-units/firewall/kyverno-capm3/*.yaml
policies, wherever each defect actually occurs — they share the same IPPool/Metal3Machine
generate-rule shape, so the same defect classes recur across them.
1. skipBackgroundRequests: false — Kyverno defaults this to true, which silently
disables the background scan generateExisting: true needs to backfill from an already-existing
IPPool.
2. kind: Policy → kind: ClusterPolicy + namespace scoping
(globalnetworkset-of-management-cluster-provisioning-addresses.yaml only):
kind: Policy+metadata.namespaceon a policy matching a cluster-scopedIPPool— the namespace field is invalid there and was silently ignored.matchhad no namespace scoping at all, so a workload cluster's provisioning pool could get mislabeledrole: management-cluster-node. Fixed by scopingmatchtonamespaces: [sylva-system].
3. RBAC — projectcalico.org/ippools on the background controller's ClusterRole
(charts/sylva-units/values.yaml) — IPPool is registered under 3 unrelated API groups on this
cluster (ipam.metal3.io — what we match; crd.projectcalico.org; projectcalico.org). Kyverno's
background controller resolves the bare kind across all registered groups when building its
resource cache, so missing RBAC on any one of the three breaks background processing cluster-wide:
ippools.projectcalico.org is forbidden: User "system:serviceaccount:kyverno:kyverno-background-controller"
cannot list resource "ippools" in API group "projectcalico.org" at the cluster scope4. Add 9120 port for metallb speaker.
5. Added an increased timeout for firewall-kyverno-capm3 to 10m ( instead of the 5 min default) and added a healthcheck for "management-cluster-addresses" config map in firewall-kyverno-capm3 to prevent firewall-common unit timeout when cm is not created in time and for a better troubleshoot.
Related to #2433
Test coverage
CI configuration
Below you can choose test deployment variants to run in this MR's CI.
Click to open to CI configuration
Legend:
| Icon | Meaning | Available values |
|---|---|---|
| Infra Provider | capd, capo, capm3 |
|
| Bootstrap Provider | kubeadm (alias kadm), rke2, okd, ck8s |
|
| Node OS | ubuntu, suse, na, leapmicro |
|
| Deployment Options | Deployment option list and description | |
| Pipeline Scenarios | Available scenario list and description | |
| Enabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Disabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Target platform | Can be used to select specific deployment environment Available platform list and description | |
| Pipeline control | autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way |
-
🎬 preview☁️ capd🚀 kadm🐧 ubuntu -
🎬 preview☁️ capo🚀 rke2🐧 suse -
🎬 preview☁️ capm3🚀 rke2🐧 ubuntu -
☁️ capd🚀 kadm🛠️ light-deploy🐧 ubuntu -
☁️ capd🚀 rke2🛠️ light-deploy🐧 suse -
☁️ capo🚀 rke2🛠️ firewall🐧 suse -
☁️ capo🚀 rke2🐧 leapmicro -
☁️ capo🚀 kadm🐧 ubuntu -
☁️ capo🚀 kadm🐧 ubuntu🟢 neuvector,mgmt:harbor -
☁️ capo🚀 rke2🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capo🚀 kadm🎬 wkld-k8s-upgrade🐧 ubuntu -
☁️ capo🚀 rke2🎬 rolling-update-no-wkld🛠️ ha🐧 suse -
☁️ capo🚀 rke2🎬 sylva-upgrade🛠️ ha🐧 ubuntu -
☁️ capo🚀 rke2🎬 sylva-upgrade-from-1.6.x🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ misc🐧 ubuntu🟢 mgmt:harbor🔴 neuvector -
☁️ capo🚀 rke2🛠️ ha,misc,openbao🐧 suse -
☁️ capo🚀 rke2🐧 suse🎬 upgrade-from-prev-tag -
☁️ capm3🚀 rke2🛠️ firewall🐧 suse -
☁️ capm3🚀 kadm🐧 ubuntu -
☁️ capm3🚀 ck8s🐧 ubuntu -
☁️ capm3🚀 kadm🎬 rolling-update-no-wkld🛠️ ha,misc🐧 ubuntu -
☁️ capm3🚀 rke2🎬 wkld-k8s-upgrade🛠️ ha🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2🎬 upgrade-from-prev-release-branch🛠️ ha🐧 suse -
☁️ capm3🚀 rke2🛠️ misc,ha🐧 suse -
☁️ capm3🚀 rke2🎬 sylva-upgrade🛠️ firewall🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 suse -
☁️ capm3🚀 ck8s🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2|okd🎬 no-update🐧 ubuntu|na -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-from-release-1.5 -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-to-main
Global config for deployment pipelines
- autorun pipelines
- allow failure on pipelines
- record sylvactl events
Notes:
- Enabling
autorunwill make deployment pipelines to be run automatically without human interaction - Disabling
allow failurewill make deployment pipelines mandatory for pipeline success. - if both
autorunandallow failureare disabled, deployment pipelines will need manual triggering but will be blocking the pipeline
Be aware: after configuration change, pipeline is not triggered automatically.
Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.