Add integration tests
Add the capability to record informer events and replay them in unit tests.
This MR comes in multiple commits, it can be relevant to look at them one by one to better understand how the the various changes were introduced, but avoid spending too much time on reviewing them individually as the code has been significantly refactored & re-organised in "Move test-env and recorder to dedicated modules" commit in order to provide a clean interface for recording, test-env management and playback of test scenarios (sorry for that, but it is very complex to re-dispatch that last change over previous commit list... I tried several times but failed to do so. I could rebuild the commit list, but it would end-up in a commit per (group of) file, which is not very different from reviewing the whole change at once.)
Here is the summary of the changes that this MR is trying to introduce:
- Add a
recorderpackage to enable the recording of the sequence of events seen by the informer in a file - Add a
testenvpackage and required infrastructure (Makefile, gitlab-ci, flux manifests...) to enable the set-up of a local API server and the replay of a recorded or forged sequence of events. - Add informer tests that replays some records and checks that the sequence of event sent by the informer matches a reference (
internal/test-env/scenarios/bootstrap-mgt-capd-referenceEventSequence.yaml) - Add informer tests playing forged scenarios and doing assertions on the internal state of various objects
- Add basic test for watch command (all commands are moved to cmd/ directory alongside with that change)
List of commits
Add event recorder
Introduce a --record option to watch command to enable to record all api events seen by the informer in a file.
For that purpose, informer stop has to be revisited to properly close the record file.
Event recorder use is tested in sylva-projects/sylva-core!3566 (closed)
Introduce envtest framework
envtest module of controller-runtime project is usually used to test operators, it starts apiserver and etcd to enable the testing of interaction with k8s API
Using that framework, we replay recorded files, and ensure that informer is maintaining the state of resources as expected.
Configure renovate to track envtest releases
Run envtest in CI
Reconfigure the CI to enable the use of envtest (using Makefile to launch tests) instead of to-be-continuous templates that are not adapted.
Save the observed sequence of object status
Each time an event is received from the informer, record relevant status of all objects (ready, reconciled, reconcilable...) and ensure that it matches with a reference that was previously observed.
Record and emulate initial API state
Add a maker in record when informer syncs, and start informer after that point when record is played back. This way, it will start in a similar state as is real cases.
Move test-env and recorder to dedicated modules
This way they will expose a cleaner API, and will be re-usable in different contexts, like testing commands for example.
Move commands and add tests
move all command files in cmd/ directory, and add basic first test for watch command.
Reorganize informer env-test
Rename scenario_test to replay_test as it better reflects what that test is doing Move to suite_test the code that can be shared with other tests
Record player improvements
Play last record even if there is no ending yaml separator Send a delayed error to client when playback is finished to avoid waiting for global timeout
Add more test scenarios
Add a couple of hand-built tests scenarios to test informer behavior:
- a basic scenario to test dependencies management
- test the support of helmRelease.spec.chartRef that was recently added