sylvactl watch goes back to previously reconciled units (calico-ready, misc-controllers)
In https://gitlab.com/sylva-projects/sylva-core/-/jobs/13008827647, we see the following:
-
(keeping in mind that
clusterunit depends onmisc-controllers-suiteunit) -
clusterunit starts reconciling at2026/02/05 23:41:59.170396 Kustomization/cluster state changed: Progressing -
at multiple times during the run the following sequence occurs:
-
the
misc-controllers-suitegoes intohealth check failed2026/02/05 23:54:15.068568 Kustomization/misc-controllers-suite state changed: HealthCheckFailed - health check failed after 30.016270739s: timeout waiting for: [Deployment/misc-controllers-suite-system/misc-controllers-suite-controller-manager status: 'InProgress'] -
sylvactl starts to wait for it instead of
cluster2026/02/05 23:54:15.072409 ╭╴Waiting for the following resources to progress: 2026/02/05 23:54:15.072418 │ Kustomization/misc-controllers-suite HealthCheckFailed - health check failed after 30.016270739s: timeout waiting for: [Deployment/misc-controllers-suite-system/misc-controllers-suite-controller-manager status: 'InProgress'] 2026/02/05 23:54:15.072421 ╰╴╴╴┄ -
some times after,
misc-controllers-suiterecovers, andsylvactl watchgoes back toclusterunit2026/02/05 23:55:15.420725 Kustomization/misc-controllers-suite state changed: ReconciliationSucceeded - Applied revision: 0.0.0-git-d2d089e1@sha256:1e3f3496c99ea01a7569311284abd25907edd903750105da95bc4389614f2861 2026/02/05 23:55:15.424543 ╭╴Waiting for the following resources to progress: 2026/02/05 23:55:15.424555 │ Kustomization/cluster DependencyNotReady - dependency 'sylva-system/cluster-node-provider-id-blacklist' is not ready 2026/02/05 23:55:15.424557 ╰╴╴╴┄
-
-
the last times this occurs, the misc-controllers-suite healthcheck is still failing while sylvactl global timeout triggers, and
sylvactl watchis only waiting for misc-controllers-suite, and hence gives this message:
2026/02/06 01:20:48.730985 Kustomization/misc-controllers-suite state changed: HealthCheckFailed - health check failed after 30.020314463s: timeout waiting for: [Deployment/misc-controllers-suite-system/misc-controllers-suite-controller-manager status: 'InProgress']
2026/02/06 01:20:48.735663 ╭╴Waiting for the following resources to progress:
2026/02/06 01:20:48.735677 │ Kustomization/misc-controllers-suite HealthCheckFailed - health check failed after 30.020314463s: timeout waiting for: [Deployment/misc-controllers-suite-system/misc-controllers-suite-controller-manager status: 'InProgress']
2026/02/06 01:20:48.735682 ╰╴╴╴┄
2026/02/06 01:21:48.755469 Kustomization/misc-controllers-suite state changed: Progressing - Reconciliation in progress
2026/02/06 01:22:18.018452 Command timeout exceeded: Kustomization/sylva-system/sylva-units-status did not became Ready after 2h7m0s
<inventory dump for Kustomization/misc-controllers-suite>
When sylvactl watch has seen a Kustomization become Ready, it is not supposed to wait again for it later in the same run, but here in this example, it does.
We've already seen this occur with calico-ready unit (on which cluster unit also depends-on).