download and verify checksum in case of multipart upload for etcd

closes #4225 (closed)

after skipping multipart upload

k get pods -n kube-system
backup-etcd-29692180-csz7k                                   0/1     Completed   0          11m
backup-etcd-29692185-bxqnd                                   0/1     Completed   0          6m10s
backup-etcd-29692190-ghc7n                                   0/1     Completed   0          70s

Server version 3.6.0
-- Etcd backed up.
-- Backup compressed.
-- Backup compressed (size: 95958291 bytes).
Added `backup` successfully.
`/tmp/tmp.PlLKjA` -> `backup/backup/kubeadm-backup_etcd_backup.tar.gz`
┌───────────┬─────────────┬──────────┬─────────────┐
│ Total     │ Transferred │ Duration │ Speed       │
│ 91.51 MiB │ 91.51 MiB   │ 00m01s   │ 74.01 MiB/s │
└───────────┴─────────────┴──────────┴─────────────┘
-- Backup uploaded
-- Size integrity check passed: actual size=95958291 bytes, s3 size uploaded=95958291 bytes
-- Multipart upload detected (ETag: 8cafc027cb99478bc660b71059213e72-6), downloading this file
`backup/backup/kubeadm-backup_etcd_backup.tar.gz` -> `/tmp/tmp.nFoKNk`
┌───────────┬─────────────┬──────────┬──────────────┐
│ Total     │ Transferred │ Duration │ Speed        │
│ 91.51 MiB │ 91.51 MiB   │ 00m00s   │ 100.10 MiB/s │
└───────────┴─────────────┴──────────┴──────────────┘
-- Checksum verification passed. DOWNLOADED_CHECKSUM: ee5e2e3179afe385c94769f91990829a LOCAL_BACKUP_CHECKSUM: ee5e2e3179afe385c94769f91990829a
Backup succeeded in 17 seconds (size: 95958291 bytes)

-- Push result to the pushgateway

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon	Meaning	Available values
☁️	Infra Provider	capd, capo, capm3
🚀	Bootstrap Provider	kubeadm (alias kadm), rke2, okd, ck8s
🐧	Node OS	ubuntu, suse, na, leapmicro
🛠️	Deployment Options	Deployment option list and description
🎬	Pipeline Scenarios	Available scenario list and description
🟢	Enabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🔴	Disabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🏗️	Target platform	Can be used to select specific deployment environment Available platform list and description
⚡	Pipeline control	autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way

• 🎬 preview ☁️ capd 🚀 kadm 🐧 ubuntu
• 🎬 preview ☁️ capo 🚀 rke2 🐧 suse
• 🎬 preview ☁️ capm3 🚀 rke2 🐧 ubuntu
• ☁️ capd 🚀 kadm 🛠️ light-deploy 🐧 ubuntu
• ☁️ capd 🚀 rke2 🛠️ light-deploy 🐧 suse
• ☁️ capo 🚀 rke2 🐧 suse 🛠️ backup
• ☁️ capo 🚀 rke2 🐧 leapmicro
• ☁️ capo 🚀 kadm 🐧 ubuntu 🛠️ backup
• ☁️ capo 🚀 kadm 🐧 ubuntu 🟢 neuvector,mgmt:harbor
• ☁️ capo 🚀 rke2 🎬 rolling-update 🛠️ ha 🐧 ubuntu
• ☁️ capo 🚀 kadm 🎬 wkld-k8s-upgrade 🐧 ubuntu
• ☁️ capo 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 suse
• ☁️ capo 🚀 rke2 🎬 sylva-upgrade 🛠️ ha 🐧 ubuntu
• ☁️ capo 🚀 rke2 🎬 sylva-upgrade-from-1.6.x 🛠️ ha,misc 🐧 ubuntu
• ☁️ capo 🚀 rke2 🛠️ ha,misc 🐧 ubuntu
• ☁️ capo 🚀 rke2 🛠️ misc 🐧 ubuntu 🟢 mgmt:harbor 🔴 neuvector
• ☁️ capo 🚀 rke2 🛠️ ha,misc,openbao:penguin: suse
• ☁️ capo 🚀 rke2 🐧 suse 🎬 upgrade-from-prev-tag
• ☁️ capm3 🚀 rke2 🐧 suse 🛠️ backup
• ☁️ capm3 🚀 kadm 🐧 ubuntu 🛠️ backup
• ☁️ capm3 🚀 ck8s 🐧 ubuntu
• ☁️ capm3 🚀 kadm 🎬 rolling-update-no-wkld 🛠️ ha,misc 🐧 ubuntu
• ☁️ capm3 🚀 rke2 🎬 wkld-k8s-upgrade 🛠️ ha 🐧 suse
• ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 ubuntu
• ☁️ capm3 🚀 rke2 🎬 upgrade-from-prev-release-branch 🛠️ ha 🐧 suse
• ☁️ capm3 🚀 rke2 🛠️ misc,ha 🐧 suse
• ☁️ capm3 🚀 rke2 🎬 sylva-upgrade 🛠️ ha,misc 🐧 suse
• ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 suse
• ☁️ capm3 🚀 ck8s 🎬 rolling-update 🛠️ ha 🐧 ubuntu
• ☁️ capm3 🚀 rke2|okd 🎬 no-update 🐧 ubuntu|na
• ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-from-release-1.5
• ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-to-main

Global config for deployment pipelines

• autorun pipelines

• allow failure on pipelines

• record sylvactl events

Notes:

• Enabling autorun will make deployment pipelines to be run automatically without human interaction
• Disabling allow failure will make deployment pipelines mandatory for pipeline success.
• if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.