Add RustFS operator

Context: #3289 (closed)

Replace MinIO with a lighter, Rust-based S3 implementation for internal object storage. RustFS is deployed via its own operator (rustfs.com/v1alpha1), which supports the same erasure-coded for tenants, TLS-enabled S3 API surface Thanos and Loki already speak.

New units:

• rustfs-operator-init - to create rustfs-operator namespace, cert-manager wiring for the operator's STS webhook , hidden and dependent on rustfs-operator
• rustfs-operator - source https://github.com/rustfs/operator.git tag 0.0.6, deployed via the operator's own Helm chart , enabled when monitoring or logging units are enabled - which installs rustfs-operator and console that is exposed via ingress. This unit is disabled by default. If this is enabled, rustfs-operator-init will be enabled along with this.

Schema additions (charts/sylva-units/values.schema.yaml):

• external_hostnames: rustfs_operator
• external_certificates.rustfs_operator: $ref cert-and-key

The Console login form expects a Kubernetes ServiceAccount bearer token. For the chart-managed Console ServiceAccount, generate a short-lived token with:

kubectl -n rustfs-system create token rustfs-operator-console --duration=24h

Future task for integration with Keycloak OIDC (Crossplane) : #4533

Documentation used : https://github.com/rustfs/operator/blob/main/deploy/rustfs-operator/README.md

Follow-ups

Upcoming MRs:

• use rustfs instead of minio for Thanos (WIP started in !9016 (merged))
• use rustfs instead of minio for Loki (started in !9016 (merged))

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon	Meaning	Available values
☁️	Infra Provider	capd, capo, capm3
🚀	Bootstrap Provider	kubeadm (alias kadm), rke2, okd, ck8s
🐧	Node OS	ubuntu, suse, na, leapmicro
🛠️	Deployment Options	Deployment option list and description
🎬	Pipeline Scenarios	Available scenario list and description
🟢	Enabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🔴	Disabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by mgmt: or wkld: to be applied only to a specific cluster type
🏗️	Target platform	Can be used to select specific deployment environment Available platform list and description
⚡	Pipeline control	autorun, manual or blocking. Can be used to override global config and start a deployment pipeline the required way

• 🎬 preview ☁️ capd 🚀 kadm 🐧 ubuntu

• 🎬 preview ☁️ capo 🚀 rke2 🐧 suse

• 🎬 preview ☁️ capm3 🚀 rke2 🐧 ubuntu

• ☁️ capd 🚀 kadm 🛠️ light-deploy 🐧 ubuntu

• ☁️ capd 🚀 rke2 🛠️ light-deploy 🐧 suse

• ☁️ capo 🚀 rke2 🟢 mgmt:rustfs-operator 🐧 suse

• ☁️ capo 🚀 rke2 🐧 leapmicro

• ☁️ capo 🚀 kadm 🐧 ubuntu

• ☁️ capo 🚀 kadm 🐧 ubuntu 🟢 neuvector,mgmt:harbor

• ☁️ capo 🚀 rke2 🎬 rolling-update 🛠️ ha 🐧 ubuntu

• ☁️ capo 🚀 kadm 🎬 wkld-k8s-upgrade 🐧 ubuntu

• ☁️ capo 🚀 rke2 🎬 rolling-update-no-wkld 🛠️ ha 🐧 suse

• ☁️ capo 🚀 rke2 🎬 sylva-upgrade 🛠️ ha 🐧 ubuntu

• ☁️ capo 🚀 rke2 🎬 sylva-upgrade-from-1.6.x 🛠️ ha,misc 🐧 ubuntu

• ☁️ capo 🚀 rke2 🛠️ ha,misc 🐧 ubuntu

• ☁️ capo 🚀 rke2 🛠️ misc 🐧 ubuntu 🟢 mgmt:harbor 🔴 neuvector

• ☁️ capo 🚀 rke2 🛠️ ha,misc,openbao 🐧 suse

• ☁️ capo 🚀 rke2 🐧 suse 🎬 upgrade-from-prev-tag

• ☁️ capm3 🚀 rke2 🐧 suse

• ☁️ capm3 🚀 kadm 🐧 ubuntu

• ☁️ capm3 🚀 ck8s 🐧 ubuntu

• ☁️ capm3 🚀 kadm 🎬 rolling-update-no-wkld 🛠️ ha,misc 🐧 ubuntu

• ☁️ capm3 🚀 rke2 🎬 wkld-k8s-upgrade 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 ubuntu

• ☁️ capm3 🚀 rke2 🎬 upgrade-from-prev-release-branch 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 rke2 🛠️ misc,ha 🐧 suse

• ☁️ capm3 🚀 rke2 🎬 sylva-upgrade 🛠️ ha,misc 🐧 suse

• ☁️ capm3 🚀 kadm 🎬 rolling-update 🛠️ ha 🐧 suse

• ☁️ capm3 🚀 ck8s 🎬 rolling-update 🛠️ ha 🐧 ubuntu

• ☁️ capm3 🚀 rke2|okd 🎬 no-update 🐧 ubuntu|na

• ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-from-release-1.5

• ☁️ capm3 🚀 rke2 🐧 suse 🎬 upgrade-to-main

Global config for deployment pipelines

• autorun pipelines
• allow failure on pipelines
• record sylvactl events

Notes:

• Enabling autorun will make deployment pipelines to be run automatically without human interaction
• Disabling allow failure will make deployment pipelines mandatory for pipeline success.
• if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.