Add integration tests

Add the capability to record informer events and replay them in unit tests.

This MR comes in multiple commits, it can be relevant to look at them one by one to better understand how the the various changes were introduced, but avoid spending too much time on reviewing them individually as the code has been significantly refactored & re-organised in "Move test-env and recorder to dedicated modules" commit in order to provide a clean interface for recording, test-env management and playback of test scenarios (sorry for that, but it is very complex to re-dispatch that last change over previous commit list... I tried several times but failed to do so. I could rebuild the commit list, but it would end-up in a commit per (group of) file, which is not very different from reviewing the whole change at once.)

Here is the summary of the changes that this MR is trying to introduce:

  • Add a recorder package to enable the recording of the sequence of events seen by the informer in a file
  • Add a testenv package and required infrastructure (Makefile, gitlab-ci, flux manifests...) to enable the set-up of a local API server and the replay of a recorded or forged sequence of events.
  • Add informer tests that replays some records and checks that the sequence of event sent by the informer matches a reference (internal/test-env/scenarios/bootstrap-mgt-capd-referenceEventSequence.yaml)
  • Add informer tests playing forged scenarios and doing assertions on the internal state of various objects
  • Add basic test for watch command (all commands are moved to cmd/ directory alongside with that change)

List of commits

Add event recorder

Introduce a --record option to watch command to enable to record all api events seen by the informer in a file.

For that purpose, informer stop has to be revisited to properly close the record file.

Event recorder use is tested in sylva-projects/sylva-core!3566 (closed)

Introduce envtest framework

envtest module of controller-runtime project is usually used to test operators, it starts apiserver and etcd to enable the testing of interaction with k8s API

Using that framework, we replay recorded files, and ensure that informer is maintaining the state of resources as expected.

Configure renovate to track envtest releases

Run envtest in CI

Reconfigure the CI to enable the use of envtest (using Makefile to launch tests) instead of to-be-continuous templates that are not adapted.

Save the observed sequence of object status

Each time an event is received from the informer, record relevant status of all objects (ready, reconciled, reconcilable...) and ensure that it matches with a reference that was previously observed.

Record and emulate initial API state

Add a maker in record when informer syncs, and start informer after that point when record is played back. This way, it will start in a similar state as is real cases.

Move test-env and recorder to dedicated modules

This way they will expose a cleaner API, and will be re-usable in different contexts, like testing commands for example.

⚠️ This commit is significantly refactoring previous ones, to that extent it may not be relevant to review changes commit by commit...

Move commands and add tests

move all command files in cmd/ directory, and add basic first test for watch command.

Reorganize informer env-test

Rename scenario_test to replay_test as it better reflects what that test is doing Move to suite_test the code that can be shared with other tests

Record player improvements

Play last record even if there is no ending yaml separator Send a delayed error to client when playback is finished to avoid waiting for global timeout

Add more test scenarios

Add a couple of hand-built tests scenarios to test informer behavior:

  • a basic scenario to test dependencies management
  • test the support of helmRelease.spec.chartRef that was recently added
Edited by Francois Eleouet

Merge request reports

Loading