Run more QCheck tests in a nightly job
We have been recently bitten by QCheck tests being flaky, for example in !3527 (merged).
The flakiness stems from QCheck using a seed to generate its data. So it sometimes generates data that shows that a test is wrong or that there's a bug. It's really cool, because it means that, as time passes and more seeds are used, more tests are being run.
However, it is annoying, because it breaks the workflow of unrelated MRs, or breaks the CI of master
.
To fix that, we should execute many more QCheck tests in a nightly pipeline (or a scheduled one as we call them in Tezos I believe), i.e. in a job where it is OK to run for much longer. It is possible to override the number of tests that QCheck executes for a single test using the ~count parameter. This issue should use this feature (or another global flag of QCheck if there is one, or introduce one in QCheck), so that we can run more tests while keeping regular CI fast. It is part of the issue to design the exact solution, consider this paragraph solely as a hint.