Skip to content

Python tests: add tests for mainnet and proposed protocol

Context

Copied from tezos/tezos!2288 (merged) as the context is the same:

We are trying to move away from proto-proposal and develop the protocol proposal in master directly. But if we modify proto_alpha in master, all tests that currently run on proto_alpha would now run on the protocol being developed instead of the protocol which is currently active on Mainnet. Instead, we would probably like to run tests on both protocols.

See tezos/tezos!2303 (merged) for an overview of this effort of which this is but one MR.

This merge request duplicates most pytests by the protocol to make this possible: one set of tests, in tests_python/tests_007 target the mainnet protocol (007 = delphi). The other set, in tests_python/tests_alpha target the protocol proposal (008 = edo = alpha).

Furthermore, the same duplication has been made for the directories tests_python/contracts:

There are two protocol specific directories:

  • contracts{_007,}
  • contracts_007 contains michelson scripts for 007
  • contracts_alpha contains michelson scripts for alpha
  • tests{_007,}
  • tests_007 contains pytest for 007
  • tests_alpha contains pytest for alpha

I didn't see the point in duplicating the directory scripts, that contains utility scripts, per protocol. So there is only scripts that works on protocol alpha (i.e. 008.)

This MR replaces tezos/tezos!2302 (closed) which was created with the wrong target branch.

To specialize the tests

In summary, this MR does the following;

  1. start from proto-proposal branch, thus containing tests for current protocol proposal edo
  2. cherry-pick !2290
  3. move contracts, tests to contracts_alpha, tests_alpha`
  4. import contracts, tests and call them contracts_007 resp tests_007 from branch master
  5. in tests_alpha and tests_007: add a protocol.py that specific functions/constants for that protocol. notably, the function activate which activates delphi and alpha for tests and tests_008 respectively.
  6. in tests_alpha and tests_007: add a contract_paths.py that contains the paths to the contracts in contracts_alpha / contracts_007 respectively.
  7. the first line of the regression logs for 007/alpha has been updated as their respective tests have been moved.
  8. run both respective test suites in the CI (this adds a heap of jobs)

Discussion:

Clearly this degree of duplication is not desirable.

  • What to do when modifying existing tests?
  • What to do when protocols are rotated?
  • What happens if the testing framework must evolve in a way that is incompatible with the current protocol?

Manually testing the MR

  1. Check out the pipeline
  2. Run tests locally
    • cd tests_python && poetry run pytest tests_alpha tests_008/
    • That's quite slow, there are now some 3000 tests... the following command timeouts slow tests and selects any marked as slow:
    • cd tests_python && poetry run pytest --timeout=30 -m "not slow" tests tests_008/
  3. verify that the contents of tests_python/{tests,contracts}_007 is the same as the contents of master:tests_python/{tests,contracts}, modulo diffs described above
  4. verify that the contents of tests_python/{tests,contracts}_alpha is the same as the contents of proto-proposal:tests_python/{tests,contracts}, modulo diffs described above

Checklist

- [ ] Document the interface of any function added or modified (see the coding guidelines) - [ ] Provide automatic testing (see the testing guide). - [ ] Add item in the Development Version section of CHANGES.md (only for new features and bug fixes).

Reviewers

@bidinger @dannywillems @paracetamolo @rafoo_

Edited by Arvid Jakobsson

Merge request reports