CI: on 'sylvactl watch' failure, dump 'sylvactl watch' final status in a non-collapsed log section (!5637) · Merge requests · Sylva-projects / sylva-core

This MR aims at making it easier to see the last lines of sylvactl watch on a CI run without having to unwrap the last relevant collapsed section and then scroll down.

How this is done:

sylvactl watch is replaced by a wrapper that, when run in CI, will tee the logs to a file
modifying the shell exit trap so that if the command that caused the exit is a sylvactl watch failure, we display the last 20 lines of the sylvactl watch output
these lines are:
- in a non-collapsed log section
- surrounded by very visible emojis to make them stand out

Example in https://gitlab.com/sylva-projects/sylva-core/-/jobs/11494579197:

❌ 🔻🔻🔻🔻🔻 Last lines of sylvactl watch... 🔎 00:00
...
2025/09/25 16:03:58.390808 Kustomization/os-images-info state changed: HealthCheckFailed - health check failed after 30.034711056s: timeout waiting for: [Job/sylva-system/os-images-info status: 'InProgress', ConfigMap/sylva-system/os-images-info status: 'NotFound']
2025/09/25 16:03:58.419739 Kustomization/os-images-info state changed: Progressing - Reconciliation in progress
2025/09/25 16:04:28.777376 Kustomization/os-images-info state changed: HealthCheckFailed - health check failed after 30.033827555s: timeout waiting for: [Job/sylva-system/os-images-info status: 'InProgress', ConfigMap/sylva-system/os-images-info status: 'NotFound']
2025/09/25 16:04:28.817867 Kustomization/os-images-info state changed: Progressing - Reconciliation in progress
2025/09/25 16:04:59.153776 Kustomization/os-images-info state changed: HealthCheckFailed - health check failed after 30.046995033s: timeout waiting for: [Job/sylva-system/os-images-info status: 'InProgress', ConfigMap/sylva-system/os-images-info status: 'NotFound']
2025/09/25 16:04:59.184327 Kustomization/os-images-info state changed: Progressing - Reconciliation in progress
2025/09/25 16:05:29.504598 Kustomization/os-images-info state changed: HealthCheckFailed - health check failed after 30.033302671s: timeout waiting for: [Job/sylva-system/os-images-info status: 'InProgress', ConfigMap/sylva-system/os-images-info status: 'NotFound']
2025/09/25 16:05:29.534501 Kustomization/os-images-info state changed: Progressing - Reconciliation in progress
2025/09/25 16:05:59.812697 Kustomization/os-images-info state changed: HealthCheckFailed - health check failed after 30.0315277s: timeout waiting for: [Job/sylva-system/os-images-info status: 'InProgress', ConfigMap/sylva-system/os-images-info status: 'NotFound']
2025/09/25 16:05:59.838301 Kustomization/os-images-info state changed: Progressing - Reconciliation in progress
2025/09/25 16:06:30.139932 Kustomization/os-images-info state changed: HealthCheckFailed - health check failed after 30.032448404s: timeout waiting for: [Job/sylva-system/os-images-info status: 'InProgress', ConfigMap/sylva-system/os-images-info status: 'NotFound']
2025/09/25 16:06:30.171545 Kustomization/os-images-info state changed: Progressing - Reconciliation in progress
2025/09/25 16:06:59.808961 Unit timeout exceeded: unit Kustomization/os-images-info did not became ready after 5m0s
Timed-out waiting for the following resources to be ready:
IDENTIFIER                                STATUS     REASON      MESSAGE
Kustomization/sylva-system/os-images-info InProgress             Kustomization generation is 1, but latest observed generation is -1
╰┄╴┬┄┄[Conditions]
   ├┄╴Reconciling                         True       Progressing Running health checks for revision 0.0.0-git-cf08f988@sha256:189a7e59abaa5bd1b1ebe51e71fdab35ae9db14c50ebe2b17c3b9260b5419e29 with a timeout of 30s
   ├┄╴Ready                               Unknown    Progressing Reconciliation in progress
   ╰┄╴Healthy                             Unknown    Progressing Running health checks for revision 0.0.0-git-cf08f988@sha256:189a7e59abaa5bd1b1ebe51e71fdab35ae9db14c50ebe2b17c3b9260b5419e29 with a timeout of 30s
❌ 🔺🔺🔺🔺🔺 🔎

Last, this MR is also adding the sylvactl watch log file to CI artifacts.

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon	Meaning	Available values
☁️	Infra Provider	`capd`, `capo`, `capm3`
🚀	Bootstrap Provider	`kubeadm` (alias `kadm`), `rke2`, `okd`, `ck8s`
🐧	Node OS	`ubuntu`, `suse`, `na`, `leapmicro`
🛠️	Deployment Options	`light-deploy`, `dev-sources`, `ha`, `misc`, `maxsurge-0`, `logging`, `no-logging`, `openbao`
🎬	Pipeline Scenarios	Available scenario list and description

Global config for deployment pipelines

autorun pipelines
allow failure on pipelines
record sylvactl events

Notes:

Enabling autorun will make deployment pipelines to be run automatically without human interaction
Disabling allow failure will make deployment pipelines mandatory for pipeline success.
if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.

Edited Sep 26, 2025 by Thomas Morin

CI: on 'sylvactl watch' failure, dump 'sylvactl watch' final status in a non-collapsed log section

CI configuration

Global config for deployment pipelines

Merge request reports