workload clusters: have "cluster" unit depend on "calico-ready" on upgrades instead of "calico"

Addresses #4240 (closed)

The motivation for this MR is to ensure that, for workload clusters, we don't end up with having cluster depend on calico on upgrades (which happens today), while what we want is calico-ready. This is already the behavior we have for the management cluster.

The key thing to change is that the ternary used now to compute calico_readiness_unit gives the reverse result:

  calico_readiness_unit: '{{ .Values._internal.state.is_upgrade | ternary "calico" "calico-ready" }}'

But to fix this computation and have cluster depend on calico-ready on upgrades, we need to fix both the ternary and ensure proper ordering of interpretation.

For mgmt cluster:

Details on the interpretation order problem

This is causing an interpretation error for workload clusters, causing cluster unit to depend on calico instead of calico-ready during upgrades:

on ⛵ management-cluster  sylva-core on  main 
❯ helm template charts/sylva-units/ \
-f charts/sylva-units/workload-cluster.values.yaml \
-f environment-values/base-capm3-virt/base/capm3-virt-values.yaml  \
-f environment-values/base-capm3-virt/base/secrets.yaml \
-f environment-values/capm3/values.yaml \
-f charts/sylva-units/test-values/workload-cluster/shared-settings-mock.values.yaml \
-f charts/sylva-units/test-values/upgrade/upgrade.yaml \
--set cluster_virtual_ip=192.168.100.10 | \
yq 'select(.kind=="Kustomization") | select(.metadata.name=="cluster") | .spec.dependsOn'
- name: calico                         # <<<<<<<<<<<<<<<<< here
- name: cluster-bmh
- name: cluster-node-provider-id-blacklist
- name: cluster-sylva-ca
- name: longhorn-instance-manager-cleanup
- name: longhorn-update-stale-replica-timeout
- name: metallb
- name: mgmt-cluster-ready
- name: misc-controllers-suite
- name: os-images-info
- name: rke2-helmchart-prevent-uninstall
- name: root-dependency-1

Since it only depends on calico, cluster and calico-ready units are reconciled in parallel, causing calico-ready timeouts since some nodes are rolled out in the meantime, causing #4240 (closed)

With proposed change, the dependency computation is fixed:

on ⛵ management-cluster  sylva-core on  fix-calico-readiness-unit-interpretation-finally 
❯ helm template charts/sylva-units/ \
-f charts/sylva-units/workload-cluster.values.yaml \
-f environment-values/base-capm3-virt/base/capm3-virt-values.yaml  \
-f environment-values/base-capm3-virt/base/secrets.yaml \
-f environment-values/capm3/values.yaml \
-f charts/sylva-units/test-values/workload-cluster/shared-settings-mock.values.yaml \
-f charts/sylva-units/test-values/upgrade/upgrade.yaml \
--set cluster_virtual_ip=192.168.100.10 | \
yq 'select(.kind=="Kustomization") | select(.metadata.name=="cluster") | .spec.dependsOn'
- name: calico-ready                      # <<<<<<<<<<<<<<<<< here
- name: cluster-bmh
- name: cluster-node-provider-id-blacklist
- name: cluster-sylva-ca
- name: longhorn-instance-manager-cleanup
- name: longhorn-update-stale-replica-timeout
- name: metallb
- name: mgmt-cluster-ready
- name: misc-controllers-suite
- name: os-images-info
- name: rke2-helmchart-prevent-uninstall
- name: root-dependency-1

Closes: #4240 (closed) Relates to: #3884 (closed)

Test coverage

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon Meaning Available values
☁️ Infra Provider capd, capo, capm3
🚀 Bootstrap Provider kubeadm (alias kadm), rke2, okd, ck8s
🐧 Node OS ubuntu, suse, na, leapmicro
🛠️ Deployment Options Deployment option list and description
🎬 Pipeline Scenarios Available scenario list and description
🟢 Enabled units Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🔴 Disabled units Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🏗️ Target platform Can be used to select specific deployment environment Available platform list and description
Pipeline control autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way
  • 🎬 preview ☁️ capd 🚀 kadm 🐧 ubuntu

  • 🎬 preview ☁️ capo 🚀 rke2 🐧 suse

  • 🎬 preview ☁️ capm3 🚀 rke2 🐧 ubuntu

  • ☁️ capd 🚀 kadm 🛠️ light-deploy 🐧 ubuntu

  • ☁️ capd 🚀 rke2 🛠️ light-deploy 🐧 suse

  • ☁️ capo 🚀 rke2 🐧 suse

  • ☁️ capo 🚀 rke2 🐧 leapmicro

  • ☁️ capo 🚀 kadm 🐧 ubuntu

  • ☁️ capo 🚀 kadm 🐧 ubuntu 🟢 neuvector,mgmt:harbor

  • ☁️ capo 🚀 rke2 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capo 🚀 kadm 🎬 wkld-k8s-upgrade 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 suse

  • ☁️ capo 🚀 rke2 🎬 sylva-upgrade 🛠️ ha 🐧 ubuntu

  • ☁️ capo 🚀 kadm 🎬 sylva-upgrade 🛠️ ha 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🎬 sylva-upgrade-from-1.6.x 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capo 🚀 rke2 🛠️ misc 🐧 ubuntu 🟢 mgmt:harbor 🔴 neuvector

  • ☁️ capo 🚀 rke2 🛠️ ha,misc,openbao🐧 suse

  • ☁️ capo 🚀 rke2 🐧 suse 🎬 upgrade-from-prev-tag

  • ☁️ capm3 🚀 rke2 🐧 suse

  • ☁️ capm3 🚀 kadm 🐧 ubuntu

  • ☁️ capm3 🚀 ck8s 🐧 ubuntu

  • ☁️ capm3 🚀 kadm 🎬 rolling-update-no-wkld 🛠️ ha,misc 🐧 ubuntu

  • ☁️ capm3 🚀 rke2 🎬 wkld-k8s-upgrade 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capm3 🚀 rke2 🎬 upgrade-from-prev-release-branch 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 rke2 🛠️ misc,ha 🐧 suse

  • ☁️ capm3 🚀 rke2 🎬 sylva-upgrade 🛠️ ha,misc 🐧 suse

  • ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 suse

  • ☁️ capm3 🚀 ck8s 🎬 rolling-update 🛠️ ha 🐧 ubuntu

  • ☁️ capm3 🚀 rke2|okd 🎬 no-update 🐧 ubuntu|na

  • ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-from-release-1.5

  • ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-to-main

Global config for deployment pipelines

  • autorun pipelines
  • allow failure on pipelines
  • record sylvactl events

Notes:

  • Enabling autorun will make deployment pipelines to be run automatically without human interaction
  • Disabling allow failure will make deployment pipelines mandatory for pipeline success.
  • if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.

Edited by Thomas Morin

Merge request reports

Loading