Add a layer between scenarios' definition and tests
Clarification and motivation
This issue is the result of a suggestion made by @diogo.castro and the resulting discussion, see here.
In cleveland we currently have several types for a scenario (e.g. ClevelandT
, ClevelandScenario
, etc.) as well as several functions to integrate these scenarios in tests (e.g. clevelandTestProp
, emulatedScenarioCaps
, etc.) be it for a unit or a property tests.
This is problematic because it doesn't separate configuring how scenarios from their definitions.
If you want to make allow a configuration change, this forces you to support and/or modify all the integration functions necessary for all the scenarios interested. In cases like #688 (closed), where we wanted to make sure that configuration cannot be changed in the middle of the scenario, this requires a lot of duplication and/or some non-trivial workarounds.
More in general, this promotes duplication across scenario types, that we should try to reduce/merge either way.
The proposed improvement is to:
use a strategy similar to hedgehog's. Hedgehog uses two different types for writing a test (
PropertyT m a
) vs configuring a test (Property
):
- you build up a new test in the
PropertyT m a
monad- you "finish" the test by wrapping it with:
property :: PropertyT IO () -> Property
- you can then add custom configuration to a
Property
.- you can integrate this
Property
into a tasty test suite withtestProperty :: TestName -> Property -> TestTree
This ensures you can only add configuration around a test, not in the middle of a test
Acceptance criteria
There is a new layer in cleveland that:
- all scenarios can be converted to and have to go trough to be executed
- can have its setting optionally modified by the options of #668 (closed) (and possibly more)
- cannot change what actions are to be executed in the scenario
- can be "finalized" to a
TestTree
or aPropertyT IO ()