Allow workload cluster to send logs in loki (!6373) · Merge requests · Sylva-projects / sylva-core

What does this MR do and why?

The aim of this MR is to fix the inability of sending workload clusters logs to loki.

Checking loki-gateway logs ( the component responsible to authentification and requests routing write/read logs) I have seen following errors:

2025/12/11 14:58:48 [error] 12#12: *16786 user "rke2-test" was not found in "/etc/nginx/secrets/.htpasswd", client: 100.72.25.212, server: , request: "POST /loki/api/v1/push HTTP/1.1", host: "loki.sylva"
100.72.25.212 - rke2-test [11/Dec/2025:14:58:48 +0000]  401 "POST /loki/api/v1/push HTTP/1.1" 179 "-" "Ruby" "192.168.16.28"

Loki gateway secret content:

k --kubeconfig ./management-cluster-kubeconfig -n loki get secret loki-gateway -o json   | jq -r '.data[".htpasswd"]' | base64 -d
mgmt-test:<mgmt-test-passwd>

And this lead to understand that the user and password configured into clusteroutput (on workload cluster) are not present into loki-gateway secret and results in not be allowed to push the logs.

loki-credentials-secret unit should create credentials and update loki-gateway once a new tenant is added, but this was not possible because, by default this unit has enabled_conditions to loki.

In workload cluster values this condition is inherited and how we do not to have loki enabled on workload cluster loki-credentials-secret will not be trigger. More properly for workload cluster is to rely on logging unit and update loki secret once a new workload cluster (having logging enabled) is created.

k --kubeconfig ./management-cluster-kubeconfig -n loki get secret loki-gateway -o json   | jq -r '.data[".htpasswd"]' | base64 -d
rke2-test:<rke2-test-passwd>
mgmt-test:<mgmt-test-passwd>

Part of issue #2553 , because for accessing all the logs from mgmt still needs to add new loki datasources to Grafana.

Test coverage

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon	Meaning	Available values
☁️	Infra Provider	`capd`, `capo`, `capm3`
🚀	Bootstrap Provider	`kubeadm` (alias `kadm`), `rke2`, `okd`, `ck8s`
🐧	Node OS	`ubuntu`, `suse`, `na`, `leapmicro`
🛠️	Deployment Options	`light-deploy`, `dev-sources`, `ha`, `misc`, `maxsurge-0`, `logging`, `no-logging`, `cilium`
🎬	Pipeline Scenarios	Available scenario list and description
🟢	Enabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by `mgmt:` or `wkld:` to be applied only to a specific cluster type
🏗️	Target platform	Can be used to select specific deployment environment (i.e `real-bmh` for capm3 )

Global config for deployment pipelines

autorun pipelines
allow failure on pipelines
record sylvactl events

Notes:

Enabling autorun will make deployment pipelines to be run automatically without human interaction
Disabling allow failure will make deployment pipelines mandatory for pipeline success.
if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.

Edited Dec 12, 2025 by Bogdan Antohe

Allow workload cluster to send logs in loki

What does this MR do and why?

Related reference(s)

Test coverage

CI configuration

Global config for deployment pipelines

Merge request reports