Introduce Sylva test framework jobs in CI (!7533) · Merge requests · Sylva-projects / sylva-core

What does this MR do and why?

This MR is a first step to introduce a test framework in our CI setup.

Today we mainly have one job per test, it's convenient but we see that it take more and more time to execute and most of this time is past to download / start container images. Moreover many of our tests are plain bash in gitlab-ci script: section, it's hard to review, impossible to lint and lead to some errors.

The proposal is to introduce pytest as a test framework, as it's relatively simple to use, based on python. It will probably not be able to cover all our usecases but it could allow to mutualize some tests under the same jobs. Benefit from python linter, Junit result export.

This MR introduce 4 new jobs to run this framework after each deployment/update phase allowing to have tests running not only at the end of the deployment.

Results are exported as Junit report

For now theses tests are non blocking as we don't want to stop an upgrade due to a slow UI for example.

Some part of the work are missing and will be added in others MRs, the actual code can be merged without breaking / changing the tests we already perform (remaining work is tracked under a common epic: sylva-projects#108 )

collaborative work from @jonathang , @sebastien.mathy and @loic.nicolle

Framework definition: sylva-projects/sylva-elements/ci-tooling/sylva-test-framework!1 (diffs)
CI image update: sylva-projects/sylva-elements/container-images/ci-image!411 (merged)

Some MRs are waiting for this one to be merged:

!7697 (merged) add a collector job
!7734 (merged) initialize a README file for this test framework

Test coverage

CI configuration

Below you can choose test deployment variants to run in this MR's CI.

Click to open to CI configuration

Legend:

Icon	Meaning	Available values
☁️	Infra Provider	`capd`, `capo`, `capm3`
🚀	Bootstrap Provider	`kubeadm` (alias `kadm`), `rke2`, `okd`, `ck8s`
🐧	Node OS	`ubuntu`, `suse`, `na`, `leapmicro`
🛠️	Deployment Options	Deployment option list and description
🎬	Pipeline Scenarios	Available scenario list and description
🟢	Enabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by `mgmt:` or `wkld:` to be applied only to a specific cluster type
🔴	Disabled units	Any available units name, by default apply to management and workload cluster. Can be prefixed by `mgmt:` or `wkld:` to be applied only to a specific cluster type
🏗️	Target platform	Can be used to select specific deployment environment Available platform list and description

Global config for deployment pipelines

autorun pipelines
allow failure on pipelines
record sylvactl events

Notes:

Enabling autorun will make deployment pipelines to be run automatically without human interaction
Disabling allow failure will make deployment pipelines mandatory for pipeline success.
if both autorun and allow failure are disabled, deployment pipelines will need manual triggering but will be blocking the pipeline

Be aware: after configuration change, pipeline is not triggered automatically. Please run it manually (by clicking the run pipeline button in Pipelines tab) or push new code.

Edited Jun 03, 2026 by Thomas Morin

Introduce Sylva test framework jobs in CI

What does this MR do and why?

Related reference(s)

Test coverage

CI configuration

Global config for deployment pipelines

Merge request reports