let sylva-capi-cluster handle BMH.spec.online, use 'lookup' to preserve state set by BMH/CAPM3 controllers

Attempt to solve #3402

This MR revisits what was done in !6583 (merged).

It reverts the addition of the Kyverno policy, to instead have sylva-capi-cluster handle the spec.online field:

• using lookup, preserve the current state if the BMH already exists
• on creation set spec.online to false by default (can be overridden via the annotation introduced in !6583 (merged))

The motivation behind this change is that #3402 seem to be due to the fact that, on upgrade from Sylva 1.6, the spec.online: true is set in the manifest stored in the cluster-bmh Helm secret, and if absent from the new manifest generated by s-c-c in main (which is the behavior after !6583 (merged)), then applying the new Helm release will result in a PATCH to remove this field -- resulting in #3402, because without this field the manifest does not conform to the schema.

This MR depends on sylva-projects/sylva-elements/helm-charts/sylva-capi-cluster!909 (merged)

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon	Meaning	Available values
☁️	Infra Provider	capd, capo, capm3
🚀	Bootstrap Provider	kubeadm (alias kadm), rke2, okd, ck8s
🐧	Node OS	ubuntu, suse, na, leapmicro
🛠️	Deployment Options	light-deploy, dev-sources, ha, misc, maxsurge-0, logging, no-logging, cilium
🎬	Pipeline Scenarios	Available scenario list and description
🟢	Enabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🏗️	Target platform	Can be used to select specific deployment environment (i.e real-bmh for capm3 )

• 🎬 preview ☁️ capd 🚀 kadm 🐧 ubuntu

• 🎬 preview ☁️ capo 🚀 rke2 🐧 suse

• 🎬 preview ☁️ capm3 🚀 rke2 🐧 ubuntu

• ☁️ capd 🚀 kadm 🛠️ light-deploy 🐧 ubuntu

• ☁️ capd 🚀 rke2 🛠️ light-deploy 🐧 suse

• ☁️ capo 🚀 rke2 🐧 suse

• ☁️ capo 🚀 rke2 🐧 leapmicro

• ☁️ capo 🚀 kadm 🐧 ubuntu

• ☁️ capo 🚀 kadm 🐧 ubuntu 🟢 neuvector,mgmt:harbor

• ☁️ capo 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 ubuntu

• ☁️ capo 🚀 kadm 🎬 wkld-k8s-upgrade 🐧 ubuntu

• ☁️ capo 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 suse

• ☁️ capo 🚀 rke2 🎬 sylva-upgrade-no-wkld 🛠️ ha 🐧 ubuntu

• ☁️ capo 🚀 rke2 🎬 sylva-upgrade-from-1.6.x 🛠️ ha,misc 🐧 ubuntu

• ☁️ capo 🚀 rke2 🛠️ ha,misc 🐧 ubuntu

• ☁️ capo 🚀 rke2 🛠️ ha,misc,openbao🐧 suse

• ☁️ capo 🚀 rke2 🐧 suse 🎬 upgrade-from-prev-tag

• ☁️ capm3 🚀 rke2 🐧 suse

• ☁️ capm3 🚀 kadm 🐧 ubuntu

• ☁️ capm3 🚀 ck8s 🐧 ubuntu

• ☁️ capm3 🚀 kadm 🎬 rolling-update-no-wkld 🛠️ ha,misc 🐧 ubuntu

• ☁️ capm3 🚀 rke2 🎬 wkld-k8s-upgrade 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 ubuntu

• ☁️ capm3 🚀 rke2 🎬 upgrade-from-prev-release-branch 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 rke2 🛠️ misc,ha 🐧 suse

• ☁️ capm3 🚀 rke2 🎬 sylva-upgrade-no-wkld 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 rke2 🎬 sylva-upgrade 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 ck8s 🎬 rolling-update 🛠️ ha 🐧 ubuntu

• ☁️ capm3 🚀 rke2|okd 🎬 no-update 🐧 ubuntu|na

• ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-from-release-1.5

• ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-to-main

Global config for deployment pipelines

• autorun pipelines
• allow failure on pipelines
• record sylvactl events

Notes:

• Enabling autorun will make deployment pipelines to be run automatically without human interaction
• Disabling allow failure will make deployment pipelines mandatory for pipeline success.
• if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.