Skip to content

Testing framework: More reasons for better negative tests ?

Rémi requested to merge testing-framework/better-fail-reasons into dev

type:changed

Problem

When writing negative tests for a contract (i.e testing that a contract fail for a given reason given some input scenario) with ligo run test. It is not always obvious to precisely identify what went wrong exactly. The current solution is the Test.transfer_* function which is returning a test_exec_result (see https://ligolang.org/docs/next/reference/test). Right now, we only have two contructors in the test_exec_failure type:

  • Reject _ contract rejection
  • Other (that wraps all other failure cases ..)

It is not obvious to "translate" all tezos errors to something that is "testable" by ligo users. Main reasons:

  1. We can't always decompile Michelson values (to Ligo values) : which means all "type based" errors would not have sufficiently presice associated data (e.g. wrong parameter passed to Tezos.transcation inside a contract)
  2. We don't (yet) have a binding from michelson to ligo source locations ? so it would be hard to test against overflow ? is that even useful ?

Solution

It is not yet perfect .. but I added a new constructor Balance_too_low of { contract_too_low : address , contract_balance : tez , spend_request : tez } which tells you if a contract failed due to an insufficient balance.

I extended the Other case to Other of string where its argument is the tezos_client error as a string (Dunno if it is a good idea .. but | Other err -> Test.log (err) is more convenient ?

Also, added more documentation on the test_exec_error types & co

Changelog details:

The type of Test.test_exec_failure has changed: a new constructor Balance_too_low is added to assert that a contract pushed an operation but did not have enough fund.

Edited by Rémi

Merge request reports