pivot timed out waiting for the MD nodes to become ready

on nightly scheduled pipelines on 9th Aprile, we observed two occurences of pivot issues:

2024/04/08 02:01:54.536462 Kustomization/sylva-system/sylva-units-status state changed: DependencyNotReady - dependency 'sylva-system/cluster' not found: Kustomization.kustomize.toolkit.fluxcd.io "cluster" not found 2024/04/08 02:27:46.317223 Command timeout exceeded Timed-out waiting for the following resources to be ready: IDENTIFIER STATUS REASON MESSAGE Kustomization/sylva-system/sylva-units-status InProgress Kustomization generation is 1, but latest observed generation is -1 ╰┄╴┬┄┄[Conditions] ╰┄╴Ready False DependencyNotReady dependency 'sylva-system/cluster' not found: Kustomization.kustomize.toolkit.fluxcd.io "cluster" not found gathering debugging logs in debug-on-exit.log file

See:

  • https://gitlab.com/sylva-projects/sylva-core/-/jobs/6568159448
  • https://gitlab.com/sylva-projects/sylva-core/-/jobs/6568001775
Assignee Loading
Time tracking Loading