Skip to content

Snoop: store full workloads

Ilias Garnier requested to merge igarnier-snoop-full-workload into master

Context

Benchmarking with snoop produces samples of the execution time of some piece of code for various workloads. Each workload is associated to nsamples samples. These results are currently not stored as is: snoop only stores a quantile (the median by default) for each workload. This prevents combining results of benchmarks performed eg in parallel. This MR lets snoop store all the samples instead of computing their median. The task of computing statistics is deferred to consumers of the benchmark data.

We take advantage of this MR to perform some cleanup and introduce new plotting functionality (plotting arbitrary quantiles of the empirical data, dumping the full timing histograms, etc). Here's a detailed menu.

Replace python matrices with a pure-ocaml solution

04867854 1ce08f3c

Use a single bigarray instead of bench-num float arrays

5d5564f7

Save full data instead of quantile

Also, remove the cull outlier command.

8e124496 86cfba79

Richer plot options

dc28f29f 6441ceac 8ecfe494 32074ad1 82f2ebb8

Clean some code around inference

f47e6587

Fix snoop tezt

c6d09bef 1f1769fa

Manually testing the MR

Checklist

  • Document the interface of any function added or modified (see the coding guidelines)
  • Document any change to the user interface, including configuration parameters (see node configuration)
  • Provide automatic testing (see the testing guide).
  • For new features and bug fixes, add an item in the appropriate changelog (docs/protocols/alpha.rst for the protocol and the environment, CHANGES.rst at the root of the repository for everything else).
  • Select suitable reviewers using the Reviewers field below.
  • Select as Assignee the next person who should take action on that MR
Edited by Ilias Garnier

Merge request reports