Skip to content

[#186] Make `morley-test` output shorter

Diogo Castro requested to merge dfacastro/#186-test-output into master

Description

Problem:

  • Some tests have very long names, they wrap around to multiple lines and make the test output hard to read.
  • We use criterion to benchmark some expressions (see !409 (merged)). But because criterion and tasty write to stdout in parallel, the test logs can be confusing and hard to read.

Solution:

  • Shorten test names where possible. Avoid naming the test after the expression being tested, as that can accidentally lead to long test names.
  • I tried using the silently package to capture criterion's output and then log it as an hedgehog footnote. However, it seems the technique in the package is not thread-safe, as sometimes it would end up swallowing some of tasty's output as well. I ended up running criterion in quiet mode instead. We already print the estimated mean time (expected upper bound & actual) when the test fails, that info should be enough.

The longest test name in morley-test is now 137 characters long (including whitespace added by tasty), which is well shorter than the width of the jupiter shell we use in the pipeline.

           calls failOrStoreAndTransfer, updates storage with an arbitrary value, updates balances, and checks pre-conditions: OK (0.03s)

The issue was not observed in the other components' test suites.

Related issue(s)

Resolves #186 (closed)

Checklist for your Merge Request

Related changes (conditional)

  • Tests (see short guidelines)

    • If I added new functionality, I added tests covering it.
    • If I fixed a bug, I added a regression test to prevent the bug from silently reappearing again.
  • Documentation

    • I checked whether I should update the docs and did so if necessary:
    • I updated changelog files of all affected packages released to Hackage if my changes are externally visible.

Stylistic guide (mandatory)

Edited by Diogo Castro

Merge request reports