Improve cis-scan-report test in CI

Loic Nicollerequested to merge
improve_cis_report_scan into main

What does this MR do and why?

Instead of failing without info when the clusterscanreport is not present, try to dump the log from the cis-operator pod before failing. This will ease the debug mainly in the case where the cis profile in use is not compatible with the actuel k8s version

eg of log in this case:

==== START logs for container cis-operator of pod cis-operator-system/cis-operator-54b4c8f6b7-sfx52 ====
time="2025-09-01T14:36:44Z" level=info msg="Starting CIS-Operator"
time="2025-09-01T14:36:44Z" level=info msg="ClusterProvider detected rke2"
time="2025-09-01T14:36:44Z" level=info msg="KubernetesVersion detected v1.32.7+rke2r1"
time="2025-09-01T14:36:44Z" level=info msg="Starting cis.cattle.io/v1, Kind=ClusterScan controller"
time="2025-09-01T14:36:44Z" level=info msg="Starting /v1, Kind=ConfigMap controller"
time="2025-09-01T14:36:44Z" level=info msg="Starting /v1, Kind=Service controller"
time="2025-09-01T14:36:44Z" level=info msg="Starting /v1, Kind=Pod controller"
time="2025-09-01T14:36:44Z" level=info msg="Starting batch/v1, Kind=Job controller"
time="2025-09-01T14:37:54Z" level=error msg="Error validating ClusterScanProfile rke2-cis-1.9-profile, error: Kubernetes version mismatch, ClusterScanProfile rke2-cis-1.9-profile is not valid for this cluster's K8s version v1.32.7+rke2r1"
time="2025-09-01T14:37:54Z" level=error msg="Error validating ClusterScanProfile rke2-cis-1.9-profile, error: Kubernetes version mismatch, ClusterScanProfile rke2-cis-1.9-profile is not valid for this cluster's K8s version v1.32.7+rke2r1"
time="2025-09-01T14:37:54Z" level=error msg="Error validating ClusterScanProfile rke2-cis-1.9-profile, error: Kubernetes version mismatch, ClusterScanProfile rke2-cis-1.9-profile is not valid for this cluster's K8s version v1.32.7+rke2r1"
==== END logs for container cis-operator of pod cis-operator-system/cis-operator-54b4c8f6b7-sfx52 ====

Related reference(s)

can help to debug in case of issue like: #2870

Test coverage

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon Meaning Available values
☁️ Infra Provider capd, capo, capm3
🚀 Bootstrap Provider kubeadm (alias kadm), rke2, okd, ck8s
🐧 Node OS ubuntu, suse, na, leapmicro
🛠️ Deployment Options light-deploy, dev-sources, ha, misc, maxsurge-0, logging, no-logging, openbao
🎬 Pipeline Scenarios Available scenario list and description

  • 🎬 preview ☁️ capd 🚀 kadm 🐧 ubuntu

  • 🎬 preview ☁️ capo 🚀 rke2 🐧 suse

  • 🎬 preview ☁️ capm3 🚀 rke2 🐧 ubuntu

  • ☁️ capd 🚀 kadm 🛠️ light-deploy 🐧 ubuntu

  • ☁️ capd 🚀 rke2 🛠️ light-deploy 🐧 suse

  • ☁️ capo 🚀 rke2 🐧 suse

  • ☁️ capo 🚀 rke2 🐧 leapmicro

  • ☁️ capo 🚀 kadm 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capo 🚀 kadm 🎬 wkld-k8s-upgrade 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 suse

  • ☁️ capo 🚀 rke2 🎬 sylva-upgrade-from-1.4.x 🛠️ ha 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🎬 sylva-upgrade-from-1.4.x 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🛠️ ha,misc,openbao🐧 suse

  • ☁️ capm3 🚀 rke2 🐧 suse

  • ☁️ capm3 🚀 kadm 🐧 ubuntu

  • ☁️ capm3 🚀 ck8s 🐧 ubuntu

  • ☁️ capm3 🚀 kadm 🎬 rolling-update-no-wkld 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capm3 🚀 rke2 🎬 wkld-k8s-upgrade 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capm3 🚀 rke2 🎬 sylva-upgrade-from-1.4.x 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 rke2 🛠️ misc,ha 🐧 suse

  • ☁️ capm3 🚀 rke2 🎬 sylva-upgrade-from-1.4.x 🛠️ ha,misc 🐧 suse

  • ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 ck8s 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capm3 🚀 rke2|okd 🎬 no-update 🐧 ubuntu|na

Global config for deployment pipelines

  • autorun pipelines
  • allow failure on pipelines
  • record sylvactl events

Notes:

  • Enabling autorun will make deployment pipelines to be run automatically without human interaction
  • Disabling allow failure will make deployment pipelines mandatory for pipeline success.
  • if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.

Merge request reports