nightly CI capm3 ck8s pipelines failing due to longhorn
CI jobs are continuously failing for capm3 ck8s for hr longhorn
- https://gitlab.com/sylva-projects/sylva-core/-/jobs/10925520492
- https://gitlab.com/sylva-projects/sylva-core/-/jobs/10909938871
- https://gitlab.com/sylva-projects/sylva-core/-/jobs/10897308638
2025/08/03 21:36:29.799032 Failed to get flux inventory for HelmRelease sylva-system/longhorn: failed to find the Helm storage object for HelmRelease 'sylva-system/longhorn': failed to get server groups: Get "https://192.168.100.2:6443/api": dial tcp 192.168.100.2:6443: i/o timeout
IDENTIFIER STATUS REASON MESSAGE
Kustomization/sylva-system/longhorn InProgress Kustomization generation is 1, but latest observed generation is -1
╰┄╴HelmRelease/sylva-system/longhorn Failed Failed to upgrade after 3 attempt(s)
├┄╴┬┄┄[Conditions]
┆ ├┄╴Stalled True RetriesExceeded Failed to upgrade after 3 attempt(s)
┆ ├┄╴Ready False UpgradeFailed Helm upgrade failed for release longhorn-system/longhorn with chart longhorn@105.2.1+up1.8.2: context deadline exceeded
┆ ╰┄╴Released False UpgradeFailed Helm upgrade failed for release longhorn-system/longhorn with chart longhorn@105.2.1+up1.8.2: context deadline exceeded
╰┄╴┬┄┄[Events]
├┄╴2025-08-03 21:05:56 Normal HelmChartCreated Created HelmChart/sylva-system/sylva-system-longhorn with SourceRef 'HelmRepository/sylva-system/unit-longhorn'
├┄╴2025-08-03 21:05:57 Normal HelmChartConfigured Configured HelmChart/sylva-system/sylva-system-longhorn with SourceRef 'HelmRepository/sylva-system/unit-longhorn'
├┄╴2025-08-03 21:10:57 Warning InstallFailed Helm install failed for release longhorn-system/longhorn with chart longhorn@105.2.1+up1.8.2: context deadline exceeded
┆ Last Helm logs:
┆
┆ 2025-08-03T21:05:57.767272803Z: creating 21 resource(s)
┆ 2025-08-03T21:05:57.960524916Z: beginning wait for 21 resources with timeout of 5m0s
┆ 2025-08-03T21:05:57.983912331Z: DaemonSet is not ready: longhorn-system/longhorn-manager. 0 out of 1 expected pods have been scheduled
┆ 2025-08-03T21:05:59.974012623Z: Deployment is not ready: longhorn-system/longhorn-driver-deployer. 0 out of 1 expected pods are ready
┆ 2025-08-03T21:10:55.973155693Z: Deployment is not ready: longhorn-system/longhorn-driver-deployer. 0 out of 1 expected pods are ready (147 duplicate lines omitted)
┆ 2025-08-03T21:10:57.962211867Z: Error received when checking status of resource longhorn-backend. Error: 'client rate limiter Wait returned an error: context deadline exceeded', Resource details: 'Resource: "/v1, Resource=services", GroupVersionKind: "/v1, Kind=Service"
┆ Name: "longhorn-backend", Namespace: "longhorn-system"'
┆ 2025-08-03T21:10:57.962216636Z: Retryable error? true
┆ 2025-08-03T21:10:57.962218229Z: Retrying as current number of retries 0 less than max number of retries 30
├┄╴2025-08-03 21:16:47 Warning UpgradeFailed Helm upgrade failed for release longhorn-system/longhorn with chart longhorn@105.2.1+up1.8.2: context deadline exceeded
┆
cc @ader1990