surprised about benches
fn execute_add(c: &mut Criterion) {
let clos = {
let mut test_vm = get_test_vm();
test_vm.program = vec![1, 0, 1, 2];
test_vm.run_once();
};
c.bench_function(
"execute_add",
move |b| b.iter(|| clos
)
);
}
Hi! The clos
has finished run_once
before perform benches, doesn't it?
I have not found such example in Criterion.rs
's doc. Can it works correctly?
how can it works like such following example?
fn execute_add(c: &mut Criterion) {
let clos = || {
let mut test_vm = get_test_vm();
test_vm.program = vec![1, 0, 1, 2];
test_vm.run_once();
};
c.bench_function(
"execute_add",
move |b| b.iter( clos)
);
}
Finally, I found your bench doesn't run.
// first
execute_add time: [0.0000 ps 0.0000 ps 0.0000 ps]
change: [-35.123% +1503.3% +5337.3%] (p = 0.39 > 0.05)
No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
// second
execute_add time: [128.81 ns 132.70 ns 137.54 ns]
change: [+9.8266% +17.976% +26.758%] (p = 0.00 < 0.05)
Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
Edited by yjhmelody