workload clusters: have "cluster" unit depend on "calico-ready" on upgrades instead of "calico"
Addresses #4240 (closed)
The motivation for this MR is to ensure that, for workload clusters, we don't end up with having cluster depend on calico on upgrades (which happens today), while what we want is calico-ready. This is already the behavior we have for the management cluster.
The key thing to change is that the ternary used now to compute calico_readiness_unit gives the reverse result:
calico_readiness_unit: '{{ .Values._internal.state.is_upgrade | ternary "calico" "calico-ready" }}'But to fix this computation and have cluster depend on calico-ready on upgrades, we need to fix both the ternary and ensure proper ordering of interpretation.
For mgmt cluster:
- the ternary order was already good
- the problem of ordering of interpretation was fixed in !7426 (merged) for mgmt cluster, but since the issue was discovered by chance while working on sylva-projects/sylva-elements/sylvactl!551 (closed), no attention was paid to the fact that we had the same pattern in
values.yaml
Details on the interpretation order problem
This is causing an interpretation error for workload clusters, causing cluster unit to depend on calico instead of calico-ready during upgrades:
on ⛵ management-cluster sylva-core on main
❯ helm template charts/sylva-units/ \
-f charts/sylva-units/workload-cluster.values.yaml \
-f environment-values/base-capm3-virt/base/capm3-virt-values.yaml \
-f environment-values/base-capm3-virt/base/secrets.yaml \
-f environment-values/capm3/values.yaml \
-f charts/sylva-units/test-values/workload-cluster/shared-settings-mock.values.yaml \
-f charts/sylva-units/test-values/upgrade/upgrade.yaml \
--set cluster_virtual_ip=192.168.100.10 | \
yq 'select(.kind=="Kustomization") | select(.metadata.name=="cluster") | .spec.dependsOn'
- name: calico # <<<<<<<<<<<<<<<<< here
- name: cluster-bmh
- name: cluster-node-provider-id-blacklist
- name: cluster-sylva-ca
- name: longhorn-instance-manager-cleanup
- name: longhorn-update-stale-replica-timeout
- name: metallb
- name: mgmt-cluster-ready
- name: misc-controllers-suite
- name: os-images-info
- name: rke2-helmchart-prevent-uninstall
- name: root-dependency-1Since it only depends on calico, cluster and calico-ready units are reconciled in parallel, causing calico-ready timeouts since some nodes are rolled out in the meantime, causing #4240 (closed)
With proposed change, the dependency computation is fixed:
on ⛵ management-cluster sylva-core on fix-calico-readiness-unit-interpretation-finally
❯ helm template charts/sylva-units/ \
-f charts/sylva-units/workload-cluster.values.yaml \
-f environment-values/base-capm3-virt/base/capm3-virt-values.yaml \
-f environment-values/base-capm3-virt/base/secrets.yaml \
-f environment-values/capm3/values.yaml \
-f charts/sylva-units/test-values/workload-cluster/shared-settings-mock.values.yaml \
-f charts/sylva-units/test-values/upgrade/upgrade.yaml \
--set cluster_virtual_ip=192.168.100.10 | \
yq 'select(.kind=="Kustomization") | select(.metadata.name=="cluster") | .spec.dependsOn'
- name: calico-ready # <<<<<<<<<<<<<<<<< here
- name: cluster-bmh
- name: cluster-node-provider-id-blacklist
- name: cluster-sylva-ca
- name: longhorn-instance-manager-cleanup
- name: longhorn-update-stale-replica-timeout
- name: metallb
- name: mgmt-cluster-ready
- name: misc-controllers-suite
- name: os-images-info
- name: rke2-helmchart-prevent-uninstall
- name: root-dependency-1Related reference(s)
Closes: #4240 (closed) Relates to: #3884 (closed)
Test coverage
CI configuration
Below you can choose test deployment variants to run in this MR's CI.
Click to open to CI configuration
Legend:
| Icon | Meaning | Available values |
|---|---|---|
| Infra Provider | capd, capo, capm3 |
|
| Bootstrap Provider | kubeadm (alias kadm), rke2, okd, ck8s |
|
| Node OS | ubuntu, suse, na, leapmicro |
|
| Deployment Options | Deployment option list and description | |
| Pipeline Scenarios | Available scenario list and description | |
| Enabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Disabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Target platform | Can be used to select specific deployment environment Available platform list and description | |
| Pipeline control | autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way |
-
🎬 preview☁️ capd🚀 kadm🐧 ubuntu -
🎬 preview☁️ capo🚀 rke2🐧 suse -
🎬 preview☁️ capm3🚀 rke2🐧 ubuntu -
☁️ capd🚀 kadm🛠️ light-deploy🐧 ubuntu -
☁️ capd🚀 rke2🛠️ light-deploy🐧 suse -
☁️ capo🚀 rke2🐧 suse -
☁️ capo🚀 rke2🐧 leapmicro -
☁️ capo🚀 kadm🐧 ubuntu -
☁️ capo🚀 kadm🐧 ubuntu🟢 neuvector,mgmt:harbor -
☁️ capo🚀 rke2🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capo🚀 kadm🎬 wkld-k8s-upgrade🐧 ubuntu -
☁️ capo🚀 rke2🎬 rolling-update-no-wkld🛠️ ha🐧 suse -
☁️ capo🚀 rke2🎬 sylva-upgrade🛠️ ha🐧 ubuntu -
☁️ capo🚀 kadm🎬 sylva-upgrade🛠️ ha🐧 ubuntu -
☁️ capo🚀 rke2🎬 sylva-upgrade-from-1.6.x🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ misc🐧 ubuntu🟢 mgmt:harbor🔴 neuvector -
☁️ capo🚀 rke2🛠️ ha,misc,openbao🐧 suse -
☁️ capo🚀 rke2🐧 suse🎬 upgrade-from-prev-tag -
☁️ capm3🚀 rke2🐧 suse -
☁️ capm3🚀 kadm🐧 ubuntu -
☁️ capm3🚀 ck8s🐧 ubuntu -
☁️ capm3🚀 kadm🎬 rolling-update-no-wkld🛠️ ha,misc🐧 ubuntu -
☁️ capm3🚀 rke2🎬 wkld-k8s-upgrade🛠️ ha🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2🎬 upgrade-from-prev-release-branch🛠️ ha🐧 suse -
☁️ capm3🚀 rke2🛠️ misc,ha🐧 suse -
☁️ capm3🚀 rke2🎬 sylva-upgrade🛠️ ha,misc🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 suse -
☁️ capm3🚀 ck8s🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2|okd🎬 no-update🐧 ubuntu|na -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-from-release-1.5 -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-to-main
Global config for deployment pipelines
- autorun pipelines
- allow failure on pipelines
- record sylvactl events
Notes:
- Enabling
autorunwill make deployment pipelines to be run automatically without human interaction - Disabling
allow failurewill make deployment pipelines mandatory for pipeline success. - if both
autorunandallow failureare disabled, deployment pipelines will need manual triggering but will be blocking the pipeline
Be aware: after configuration change, pipeline is not triggered automatically.
Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.