loki-credentials-secret unit stuck in creation when loki unit is disabled
Summary
When loki unit is disabled explicitly and logging is enabled then loki-credentials-secret unit creation fails and could hamper the deployment.
Steps to reproduce
To reproduce it disable the loki unit while logging unit is enabled.
- After reviewing the Flux controller logs and the
loki-credentials-secretKustomization, it was found that the Kustomization is failing due to a failed health check on theloki-secretsresource. Additionally, theloki-secretsresource is missing in the sylva-system namespace, which is causing both the health check failure and the failure of theloki-credentials-secretKustomization. - Secret
loki-secretsis created by kyverno policy (clusterpolicy)loki-aggregated-secret. This clusterpolicy was also not there. - Also
lokikustomization unit is itself disabled. If the unit is disabled we might also need to disable thisloki-credentials-secret. -
loggingunit is enabled in the cluster andlokiunit's enabled_condition depends onloggingunit. Ifloggingis enabledlokiwill be enabled. Disablinglokiexplicitly would raise the issue which we are facing currently.
What is the current bug behavior?
Unit loki-credentials-secret remains in failed state.
What is the expected correct behavior?
Unit loki-credentials-secret should not be created when loki unit is disabled.
Relevant logs and/or screenshots
$ kubectl get ks -A | grep logging
sylva-system logging 6d4h True Applied revision: 1.4.1@sha256:f83bdeecf7d955e10814139c1c5172f8cbc0a984d2740d73bb4cd6170de29806
sylva-system logging-config 6d4h True Applied revision: 1.4.1@sha256:f83bdeecf7d955e10814139c1c5172f8cbc0a984d2740d73bb4cd6170de29806
sylva-system logging-crd 6d4h True Applied revision: 1.4.1@sha256:f83bdeecf7d955e10814139c1c5172f8cbc0a984d2740d73bb4cd6170de29806
wc1 logging 3d4h False dependency 'wc1/calico-ready' is not ready
wc1 logging-config 3d4h False dependency 'wc1/calico-ready' is not ready
wc1 logging-crd 3d4h False dependency 'wc1/calico-ready' is not ready
$ kubectl get ks -A | grep loki
sylva-system loki-credentials-secret 6d4h False health check failed after 30.028380884s: timeout waiting for: [Secret/sylva-system/loki-secrets status: 'NotFound']
sylva-system sylva-units-status 6d4h False dependency 'sylva-system/loki-credentials-secret' is not ready
cc: @feleouet @claudineLM
Edited by Nitin Sharma