Skip to content

Benchmarks/watch_regressions: add a file with a selection of comparisons

Nicolas Ayache requested to merge naih@watch_regressions-selection into master

Context

When receiving feedback after a run of the benchmarks, we get a report, all.csv, with the value of all parameters on all runs.

We also get an alert file with the list of parameters with a regression. We can then check the evolution of the value of these specific parameters in all.csv, for analysis.

This request adds a report, selected.csv, that filters all.csv with only the lines with a parameter with a regression. That removes the small burden of searching in all.csv and speeds up analyses a bit.

Manually testing the MR

After downloading and extracting this set of data, and applying this patch that removes Slack messaging and AWS interactions, applies the fixes from !7824 (merged), and sets variables according to my configuration (adjusting OCTEZ_DIR and INPUT_CSV_DIR should be enough), then running watch_regressions leads to this report, where we can see a reduced selection of parameters.

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Nicolas Ayache

Merge request reports