push-helm-artifacts: add ability to disambiguate when two assets exist for the same (chart,version), use it for rke2-multus
Closes #4989 (closed)
Please see in #4989 (closed) the details about the context and why this being added.
This MR does the following:
- evolve
push-helm-chart-artifacts.pyto let it pick a specific Helm chart archive when the Helm repo index has more than one for a given chart:version- the tool now accepts an optional
DEDUP_HELM_CHART_IGNORE_URL, when set the index items that point to an archive matching this regex will be ignore - this is used only in the presence of duplicates
- the "download archive" step in such a case will use a plain Python urllib.request instead of calling
helm pull
- the tool now accepts an optional
- in the absence of duplicate items, nothing changes,
helm pullis still used - another addition was made to the tool to make it easy to test: the
HELM_CHART_FILTERenvironment variable can be set to process only a subet of Helm charts (see example below) - to address the specific problem we have with
rke2-multus-1.34-1.36, update GitLab CI job to use the new feature
Testing
This MR was tested with something reflecting the problem with rke2-multus Helm chart which lead to #4989 (closed) :
$ DEDUP_HELM_CHART_IGNORE_URL='1.34-1.36' \
PUSH_HELM_ARTIFACTS_MODE=verify \
HELM_CHART_FILTER='multus$' \
tools/oci/push-helm-charts-artifacts.py
2026-11-13 11:52:05,259 [INFO] (working in /tmp/user/1000/sylva-artifact-5d0wk8f_)
--- processing unit multus
2026-11-13 11:52:05,819 [INFO] - processing version v4.3.102 from sylva-units values
2026-11-13 11:52:05,819 [INFO] fetching index.yaml from https://rke2-charts.rancher.io/index.yaml
2026-11-13 11:52:07,735 [INFO] index.yaml has 2 entries for rke2-multus:v4.3.102 - deduplicating with ignore-url regex '1.34-1.36'
2026-11-13 11:52:07,735 [INFO] ignoring entry with urls: ['assets/rke2-multus-1.34-1.36/rke2-multus-v4.3.102.tgz']
2026-11-13 11:52:07,735 [INFO] downloading https://rke2-charts.rancher.io/assets/rke2-multus/rke2-multus-v4.3.102.tgz -> /tmp/user/1000/sylva-artifact-5d0wk8f_/rke2-multus-v4.3.102.tgz
2026-11-13 11:52:07,991 [INFO] Checking if OCI artifact exists: rke2-multus:v4.3.102 :: oci://registry.gitlab.com/sylva-projects/sylva-core/rke2-multus
2026-11-13 11:52:07,991 [INFO] running: helm pull oci://registry.gitlab.com/sylva-projects/sylva-core/rke2-multus --version v4.3.102 -d /tmp/user/1000/sylva-pulled-jloygn4e
2026-11-13 11:52:09,816 [WARNING] !stderr: Pulled: registry.gitlab.com/sylva-projects/sylva-core/rke2-multus:v4.3.102
2026-11-13 11:52:09,816 [WARNING] !stderr: Digest: sha256:256c4bf0f6c11b4fdd80c8e1e9b23e7ee15eeb913cfb565b65a71170300eb8e2
2026-11-13 11:52:09,817 [INFO] Checking the integrity of the existing artifact rke2-multus:v4.3.102 :: oci://registry.gitlab.com/sylva-projects/sylva-core/rke2-multus
2026-11-13 11:52:09,817 [INFO] running: tar -xzf /tmp/user/1000/sylva-artifact-5d0wk8f_/rke2-multus-v4.3.102.tgz -C /tmp/user/1000/tgz-eq9oymyg
2026-11-13 11:52:09,822 [INFO] running: tar -xzf /tmp/user/1000/sylva-pulled-jloygn4e/rke2-multus-v4.3.102.tgz -C /tmp/user/1000/sylva-pulled-jloygn4e
2026-11-13 11:52:09,826 [INFO] ---------- make a diff --------------
2026-11-13 11:52:09,826 [INFO] running: diff -wur --exclude Chart.lock --exclude .git /tmp/user/1000/tgz-eq9oymyg/ /tmp/user/1000/sylva-pulled-jloygn4e/
2026-11-13 11:52:09,828 [INFO] Integrity check: okThis new bit in logs is showing that this MR does what we want:
index.yaml has 2 entries for rke2-multus:v4.3.102 - deduplicating with ignore-url regex '1.34-1.36'
ignoring entry with urls: ['assets/rke2-multus-1.34-1.36/rke2-multus-v4.3.102.tgz']Development
Development was assisted by OpenCode/Claude Sonnet 4.6.
CI configuration
Below you can choose test deployment variants to run in this MR's CI.
Click to open to CI configuration
Legend:
| Icon | Meaning | Available values |
|---|---|---|
| Infra Provider | capd, capo, capm3 |
|
| Bootstrap Provider | kubeadm (alias kadm), rke2, okd, ck8s |
|
| Node OS | ubuntu, suse, na, leapmicro |
|
| Deployment Options | Deployment option list and description | |
| Pipeline Scenarios | Available scenario list and description | |
| Enabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Disabled units | Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type |
|
| Target platform | Can be used to select specific deployment environment Available platform list and description | |
| Workload deployment workflow | legacy (default; direct workload-cluster apply) or gitops (deploy workload cluster through the GitOps-managed flow) |
|
| Pipeline control | autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way |
-
🎬 preview☁️ capd🚀 kadm🐧 ubuntu -
🎬 preview☁️ capo🚀 rke2🐧 suse -
🎬 preview☁️ capm3🚀 rke2🐧 ubuntu -
☁️ capd🚀 kadm🛠️ light-deploy🐧 ubuntu -
☁️ capd🚀 rke2🛠️ light-deploy🐧 suse -
☁️ capo🚀 rke2🐧 suse -
☁️ capo🚀 rke2🐧 leapmicro -
☁️ capo🚀 kadm🐧 ubuntu -
☁️ capo🚀 kadm🐧 ubuntu🔀 gitops -
☁️ capo🚀 kadm🐧 ubuntu🟢 neuvector,mgmt:harbor -
☁️ capo🚀 rke2🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capo🚀 kadm🎬 wkld-k8s-upgrade🐧 ubuntu -
☁️ capo🚀 rke2🎬 rolling-update-no-wkld🛠️ ha🐧 suse -
☁️ capo🚀 rke2🎬 sylva-upgrade🛠️ ha🐧 ubuntu -
☁️ capo🚀 rke2🎬 sylva-upgrade-from-1.6.x🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2🛠️ ha,misc🐧 ubuntu -
☁️ capo🚀 rke2|okd🎬 no-update🐧 ubuntu|na -
☁️ capo🚀 rke2🛠️ misc🐧 ubuntu🟢 mgmt:harbor🔴 neuvector -
☁️ capo🚀 rke2🛠️ ha,misc,openbao🐧 suse -
☁️ capo🚀 rke2🐧 suse🎬 upgrade-from-prev-tag -
☁️ capm3🚀 rke2🐧 suse -
☁️ capm3🚀 kadm🐧 ubuntu -
☁️ capm3🚀 rke2🛠️ external-openbao-no-tls🐧 ubuntu -
☁️ capm3🚀 rke2🛠️ external-openbao🐧 suse -
☁️ capm3🚀 kadm🛠️ external-openbao,openbao-namespace🐧 ubuntu -
☁️ capm3🚀 ck8s🐧 ubuntu -
☁️ capm3🚀 kadm🎬 rolling-update-no-wkld🛠️ ha,misc🐧 ubuntu -
☁️ capm3🚀 rke2🎬 wkld-k8s-upgrade🛠️ ha🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2🎬 upgrade-from-prev-release-branch🛠️ ha🐧 suse -
☁️ capm3🚀 rke2🛠️ misc,ha🐧 suse -
☁️ capm3🚀 rke2🎬 sylva-upgrade🛠️ ha,misc🐧 suse -
☁️ capm3🚀 kadm🎬 rolling-update🛠️ ha🐧 suse -
☁️ capm3🚀 ck8s🎬 rolling-update🛠️ ha🐧 ubuntu -
☁️ capm3🚀 rke2|okd🎬 no-update🐧 ubuntu|na -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-from-release-1.5 -
☁️ capm3🚀 rke2🐧 suse🎬 upgrade-to-main
Global config for deployment pipelines
- autorun pipelines
- allow failure on pipelines
- record sylvactl events
Notes:
- Enabling
autorunwill make deployment pipelines to be run automatically without human interaction - Disabling
allow failurewill make deployment pipelines mandatory for pipeline success. - if both
autorunandallow failureare disabled, deployment pipelines will need manual triggering but will be blocking the pipeline
Be aware: after configuration change, pipeline is not triggered automatically.
Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.