Testing Framework POC Part 2
This extends the jsonnet-tool test
testing framework substantially, based on the feedback from attempting to retrofit this tool to GitLab Dedicated's Instrumentor project, which has a very large test suite.
- Adds two new matchers, one for plain text, one for arbitrary JSON not backed by a fixture file.
- Moves test bodies to a
function()
closure. This allows control of which tests are run, as well as allowing timing of manifestation to be measures. - Treats manifestation as a precursor to matching. This is how it was happening under the hood previously, but not it allows the speed of each test to be understood.
- Replaces the diff library with something easier to read
- Consumes traces and only emits the traces for failed tests (by default, there's a switch to fix this), The trace is displayed after the test failure message.
- Adds some consistency in the naming of Visitor event function names
- Adds a base class for visitors, so that each visitor only needs to implement the methods it will override.
See https://gitlab.com/gitlab-com/gl-infra/gitlab-dedicated/instrumentor/-/merge_requests/2636 for an example of this testing tool in action.
Demo
Edited by Andrew Newdigate