use of Kustomization.spec.healthCheckExprs before Flux upgraded to 2.5.x
We have some sylva upgrade runs that fail with:
2025/05/12 18:09:19.438400 HelmChart/sylva-system/sylva-system-postgres has been deleted
2025/05/12 18:09:35.585267 HelmRelease/sylva-units state changed: StateError - Could not determine release state: unable to determine cluster state:
Kustomization/sylva-system/minio-monitoring dry-run failed:
failed to create typed patch object (sylva-system/minio-monitoring; kustomize.toolkit.fluxcd.io/v1, Kind=Kustomization):
.spec.healthCheckExprs: field not declared in schema
2025/05/12 18:09:58.644200 HelmRelease/sylva-units state changed: Progressing - correcting cluster drift
2025/05/12 18:10:11.356741 HelmRelease/sylva-units state changed: ArtifactFailed - Source not ready: artifact not found. Retrying in 8s
2025/05/12 18:10:32.715812 HelmRelease/sylva-units state changed: Progressing - reconciliation in progress
2025/05/12 18:10:32.854733 HelmRelease/sylva-units state changed: Progressing - correcting cluster drift
Timed-out waiting for the following resources to be ready:
* HelmRelease/sylva-units: Progressing - correcting cluster drift
2025/05/12 18:10:40.378279 Command timeout exceeded
(https://gitlab.com/sylva-projects/sylva-core/-/jobs/10008509970)
This happens because sylva-core main uses the spec.healthCheckExprs field of Kustomization that only exists in Flux 2.5.x, and on an upgrade from Sylva 1.3 sylva-units can't set this field yet, because Flux is still in Flux 2.4.
The subtelty here is that this issue does not always occurs:
- when sylva-units apply the new Kustomization manifest, nothing "breaks" (Helm SDK let's us use a non-existing field)
- the error above only occurs on the failed drift correction that happens as a subsequent check
- if Flux is upgraded quickly enough we don't see this error
Edited by Thomas Morin