Rename and revisit _internal.sylva_namespaces computation and tests
What does this MR do and why?
What
What I'm proposing as actual changes to current behavior, on top of the _internal value rename is:
- keep static list part of
._internal.sylva_namespaces_not_labeled_by_fluxfor only operator-owned namespaces and avoid including K8s initial namespaces likedefault&kube-system - have the python test exempt known (k8s initial, rancher/fleet ad-hoc, w-c-o) namespaces and thus allow the test to check that all others have the
sylva.org/sylva-managed: "true"label.
If you think the default & kube-system need to be kept in ._internal.sylva_namespaces_not_labeled_by_flux I can revert this.
Why
Learnt during this MR's initial state review that ._internal.sylva_namespaces is a small complement to all namespaces (manifests) deployed by Flux (getting sylva.org/sylva-managed: "true" labels directly via commonMetadata.labels), covering only HelmRelease.spec.targetNamespace from units with HelmRelease.spec.install.createNamespace: true and the static list of operator-created (CNI and Rancher only AFAICS) namespaces, not a derivation of "every ns sylva-units deploys".
What it does
-
Renames the
._internal.sylva_namespacesto._internal.sylva_namespaces_not_labeled_by_fluxto make that contract explicit and adds a doc comment to mentioncommonMetadata.labelsimplicit injection by Flux controllers. -
Tries to improve this
._internal.sylva_namespaces_not_labeled_by_flux(currently feeding the namespace allow-list consumed by the sylva-namespaces-label Kyverno policy) so it is more accurate by:2.1 conditioning the namespace generated by Calico CNI (vs Cilium which is deployed in
kube-system, not an operator-created ns, but part of the https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#initial-namespaces), droppingtigera-operatorns from the static list because it seems it's handled by Flux directly:capo-mgmt|default] ☸ capo-mgmt ➜ k get ns tigera-operator --show-managed-fields -o json \ | jq -r '.metadata.managedFields[] | select(.fieldsV1["f:metadata"]["f:labels"]["f:sylva.org/sylva-managed"]) | .manager' helm-controller [capo-mgmt|default] ☸ capo-mgmt ➜ k get ns calico-system --show-managed-fields -o json \ | jq -r '.metadata.managedFields[] | select(.fieldsV1["f:metadata"]["f:labels"]["f:sylva.org/sylva-managed"]) | .manager' background-controller2.2 removing
kube-systemfrom the static list due to the fact it's part of https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#initial-namespaces and I believe the static part of._internal.sylva_namespaces_not_labeled_by_fluxshould be for operators-owned ns only. Also, until now it was labeled by Flux but only in some situations, mainly clusters wherelocal-path-provisionerwas not specifically disabled, due tolocal-path-provisionerunit having it asHelmRelease.spec.targetNamespace, withHelmRelease.install.createNamespace: true. Sincekube-systemis already present on the cluster, I took the decision to dropunits.local-path-provisioner.helmrelease_spec.install.createNamespace: true2.3 similarly to
kube-system, thedefaultnamespace is also part of https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/#initial-namespaces, so I dropped it from the static list following the same reasoning.2.4 removing
cattle-resources-systemas it's created by Flux and labeled viacommonMetadata.labels, unitrancher-backup-restore-operator-init2.5 removing
cattle-dashboardsas it's created by Flux and labeled viacommonMetadata.labels, frommonitoringunit'srancher-monitoringhelm template, given default chart values:defaultDashboards: namespace: cattle-dashboards useExistingNamespace: falseNote 1: the
localnamespace is Rancher-generator, so it remains in the static list (dedicated to operators-owned ns):[capo-mgmt|default] ☸ capo-mgmt ➜ kubectl get ns local -o jsonpath='{.metadata.ownerReferences}' | jq . [ { "apiVersion": "management.cattle.io/v3", "kind": "Cluster", "name": "local", "uid": "1c8fa3c8-daa2-459a-8c43-b316a12e4187" } ]Note 2: even though it is an operator-owned ns, we're not considering
workload-cluster-operatorper-workload cluster namespaces(<team>--<cluster>)because we can't predict their names, since team/cluster definitions can arrive via theworkload-teams-repoGitOps path, so that would not be available for s-u at render time. -
Refactors
test_sylva_namespaces_are_labeledtest, which was before expected to fail (due to https://docs.pytest.org/en/stable/how-to/skipping.html#xfail-mark-test-functions-as-expected-to-fail use) due to existing Sylva CI namespaces that can't be expected to be labelled- K8s initial namespaces:
default,kube-system,kube-public,kube-node-lease - Rancher and Fleet ns: c-, p-, c--p-, cluster-fleet-*
- workload-cluster-operator ns for each workload cluster
- K8s initial namespaces:
We now, post refactoring, have exemptions for the above 3 items, and are able to determine a real miss in our ._internal.sylva_namespaces_not_labeled_by_flux logic if any namespace is found missing sylva.org/sylva-managed: "true" label.
Additionally, we introduce another test_expected_namespaces_carry_label test that is checking the 2 labelling paths (Flux's own commonMetadata.labels and the sylva-namespaces-label Kyverno policy) work, by confirming the sylva.org/sylva-managed: "true" label presence when the ns is either:
- managed by Flux - the unit is Kustomization/HelmRelease (ns would have the another label key like
kustomize.toolkit.fluxcd.io/nameorhelm.toolkit.fluxcd.io/name) - listed in
_internal.sylva_namespaces_not_labeled_by_flux
Related reference(s)
Test coverage
CI configuration
Below you can choose test deployment variants to run in this MR's CI.
Click to open to CI configuration
Legend:
| Icon | Meaning | Available values |
|---|---|---|
| Infra Provider | capd, capo, capm3 |
|
| Bootstrap Provider | kubeadm (alias kadm), rke2, okd, ck8s |
|
| Node OS | ubuntu, suse, na, leapmicro |
|
| Deployment Options | Deployment option list and description | |
| Pipeline Scenarios | Available scenario list and description | |
| Enabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Disabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Target platform | Can be used to select specific deployment environment Available platform list and description | |
| Pipeline control | autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way |
-
🎬 preview☁️ capd🚀 kadm🐧 ubuntu -
🎬 preview☁️ capo🚀 rke2🐧 suse -
🎬 preview☁️ capm3🚀 rke2🐧 ubuntu -
☁️ capd🚀 kadm🛠️ light-deploy🐧 ubuntu -
☁️ capd🚀 rke2🛠️ light-deploy🐧 suse -
☁️ capo🚀 rke2🐧 suse -
☁️ capo🚀 rke2🐧 leapmicro -
☁️ capo🚀 kadm🐧 ubuntu -
☁️ capo🚀 kadm🐧 ubuntu🟢 neuvector,mgmt:harbor -
☁️ capo🚀 rke2🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capo🚀 kadm🎬 wkld-k8s-upgrade🐧 ubuntu -
☁️ capo🚀 rke2🎬 rolling-update-no-wkld🛠️ ha🐧 suse -
☁️ capo🚀 rke2🎬 sylva-upgrade🛠️ ha🐧 ubuntu -
☁️ capo🚀 rke2🎬 sylva-upgrade-from-1.7.x🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ misc🐧 ubuntu🟢 mgmt:harbor🔴 neuvector -
☁️ capo🚀 rke2🛠️ ha,misc,openbao🐧 suse -
☁️ capo🚀 rke2🐧 suse🎬 upgrade-from-prev-tag -
☁️ capm3🚀 rke2🐧 suse -
☁️ capm3🚀 kadm🐧 ubuntu -
☁️ capm3🚀 ck8s🐧 ubuntu -
☁️ capm3🚀 kadm🎬 rolling-update-no-wkld🛠️ ha,misc🐧 ubuntu -
☁️ capm3🚀 rke2🎬 wkld-k8s-upgrade🛠️ ha🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2🎬 upgrade-from-prev-release-branch🛠️ ha🐧 suse -
☁️ capm3🚀 rke2🛠️ misc,ha🐧 suse -
☁️ capm3🚀 rke2🎬 sylva-upgrade🛠️ ha,misc🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 suse -
☁️ capm3🚀 ck8s🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2|okd🎬 no-update🐧 ubuntu|na -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-from-release-1.5 -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-to-main
Global config for deployment pipelines
- autorun pipelines
- allow failure on pipelines
- record sylvactl events
Notes:
- Enabling
autorunwill make deployment pipelines to be run automatically without human interaction - Disabling
allow failurewill make deployment pipelines mandatory for pipeline success. - if both
autorunandallow failureare disabled, deployment pipelines will need manual triggering but will be blocking the pipeline
Be aware: after configuration change, pipeline is not triggered automatically.
Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.