Find a better to way to organize tests and integrate with `tasty`
Clarification and motivation
In #53 (closed) we are merging nettest and integrational interfaces for writing tests. As result, we will have only nettest interface that allows you to run tests in emulated environment and in real network.
As result, it becomes harder to integrate these tests with tasty
and organize their high-level structure. For example, you can observe it in !569 (merged). Our old tests are grouped into TestTree
s using standard tasty
functions. But in new nettests we just run all tests one by one and separate them by niComment
. When we run them via Pure implementation, we only get a single line from tasty
that says whether everything is alright. When we run them via Client implementation, we apparently don't use tasty
at all. Ideally we want to have granular tests, not stop on failure and get nice output and other tasty
features (like xml that gitlab is able to parse).
Another topic that comes to my mind is whether it's a good idea to have two executables (test-suites). I think it should be possible to run only pure tests or non-pure (i. e. talking to a node) tests as well. But instead of having two executables we can have a flag that controls whether we run impure tests. That might be simpler than having two test-suites with similar logic.
I am now looking at
, testCase "Key update with good threshold is accepted" $
nettestTestExpectation $ uncapsNettest $ do
let
counter = 0
threshold = 2
(msig, _) <- prepareContracts counter threshold masterPKList
let
action = ChangeKeys (3, Generic.Keys masterPKList)
bs = bsToSign msig counter action
call msig (Call @"MainParameter") $ mkMultisigParam counter action $
Generic.Signatures $
Nothing : Prelude.map (\sk -> Just $ SignatureEd25519 $ Ed22519.sign sk bs)
[bobSK, carlosSK]
One possible approach that comes to my mind is to:
- Add a flag specifying whether tests should be launched via Client implementation.
- Replace
nettestTestExpectation
with something that takes this flag as an argument and either behaves as it behaves now (i. e. returnsAssertion
) or returns a tree containing two tests (one runs pure implementation, one runs impure).
But I am not sure if it's a good idea.
Acceptance criteria
The topics mentioned above are discussed, conclusions are made, we have a way to organize tests that we consider the best one.