upgrade from 1.1.1 - timeout updating HelmRelease/sylva-units during "Trigger reconciliation of units"

Jobs upgrading from 1.1.1 are often (very often, maybe always) stuck on updating sylva-units/HelmRelease during "Trigger reconciliation of units" step.

Example: https://gitlab.com/sylva-projects/sylva-core/-/jobs/7635875038#L579

🎯 Trigger reconciliation of units
trigger reconciliation of sylva-units HelmRelease...
  helmrelease.helm.toolkit.fluxcd.io/sylva-units annotated
2024/08/21 20:47:51.918891 GitRepository/sylva-core                           Succeeded - stored artifact for revision 'sha1:d12ed61cb40b3627eb096bcfb02fb687e0cca687'
2024/08/21 20:47:51.918927 HelmChart/sylva-system-sylva-units                 ChartPackageSucceeded - packaged 'sylva-units' chart with version '0.0.0-git+d12ed61cb40b.1' and merged values files [charts/sylva-units/values.yaml charts/sylva-units/management.values.yaml]
2024/08/21 20:47:51.918932 HelmRelease/sylva-units                            Progressing - Running 'upgrade' action with timeout of 5m0s
2024/08/21 20:48:46.967658 Kustomization/sylva-system/root-dependency-2 has been deleted
2024/08/21 20:48:47.536402 HelmRelease/sylva-units state changed: Updated - ObservedGeneration is 2 whereas Generation is 4
2024/08/21 20:48:47.536458  ╭╴Waiting for the following resources to progress:
2024/08/21 20:48:47.536523  │  HelmRelease/sylva-units               Updated - ObservedGeneration is 2 whereas Generation is 4
2024/08/21 20:48:47.536529  ╰╴╴╴┄
2024/08/21 20:48:47.953800 HelmRelease/sylva-units state changed: Updated - ObservedGeneration is 3 whereas Generation is 4
2024/08/21 20:48:49.017522 HelmChart/sylva-system-sylva-units state changed: Updated - ObservedGeneration is 1 whereas Generation is 2
2024/08/21 20:48:49.017592  ╭╴Waiting for the following resources to progress:
2024/08/21 20:48:49.017606  │  HelmChart/sylva-system-sylva-units               Updated - ObservedGeneration is 1 whereas Generation is 2
2024/08/21 20:48:49.017614  ╰╴╴╴┄
2024/08/21 20:48:49.138697 HelmRelease/sylva-units state changed: HelmChartNotReady - HelmChart 'sylva-system/sylva-system-sylva-units' is not ready: latest generation of object has not been reconciled
2024/08/21 20:48:49.185835 HelmRelease/sylva-units state changed: HelmChartNotReady - HelmChart 'sylva-system/sylva-system-sylva-units' is not ready: latest generation of object has not been reconciled
2024/08/21 20:48:49.329345 HelmChart/sylva-system-sylva-units state changed: Progressing - building artifact: packaged 'sylva-units' chart with version '0.0.0-git+9749055fa531.2' and merged values files [charts/sylva-units/values.yaml charts/sylva-units/management.values.yaml]
2024/08/21 20:48:49.428518 HelmChart/sylva-system-sylva-units state changed: Updated - ObservedGeneration is 1 whereas Generation is 2
2024/08/21 20:48:49.466218 HelmChart/sylva-system-sylva-units state changed: Updated - lastHandledReconcileAt is 2024-08-21T19:55:42 whereas reconcileRequestedAt is 2024-08-21T20:48:49
2024/08/21 20:48:49.681933 HelmChart/sylva-system-sylva-units state changed: ChartPackageSucceeded - packaged 'sylva-units' chart with version '0.0.0-git+9749055fa531.2' and merged values files [charts/sylva-units/values.yaml charts/sylva-units/management.values.yaml]
2024/08/21 20:48:49.681989  ╭╴Waiting for the following resources to progress:
2024/08/21 20:48:49.681998  │  HelmRelease/sylva-units               HelmChartNotReady - HelmChart 'sylva-system/sylva-system-sylva-units' is not ready: latest generation of object has not been reconciled
2024/08/21 20:48:49.682003  ╰╴╴╴┄
2024/08/21 20:48:49.986599 HelmRelease/sylva-units state changed: Progressing - Running 'upgrade' action with timeout of 5m0s
Timed-out waiting for the following resources to be ready:
    * HelmRelease/sylva-units: Progressing - Running 'upgrade' action with timeout of 5m0s

Checking the artifacts show that the HelmRelease is fully reconciled.

/cc @feleouet

Edited Aug 22, 2024 by Thomas Morin
Assignee Loading
Time tracking Loading