Skip to content

Use benchmark-ips to measure performance

Peter Leitzen requested to merge pl-benchmark-ips into main

Description

Now it's more convenient to compare performance between branches.

For example:

# Run on main
bundle exec ruby benchmarks/repeated_invocation.rb

git checkout my-perf-branch

# Run benchmarks on perf branch and automatically compare!
bundle exec ruby benchmarks/repeated_invocation.rb

Example output

Warming up --------------------------------------
cached - known ability - pl-benchmark-ips
                        48.148k i/100ms
cached - unknown ability - pl-benchmark-ips
                        49.982k i/100ms
uncached - known ability - pl-benchmark-ips
                         1.476k i/100ms
uncached - unknown ability - pl-benchmark-ips
                        10.571k i/100ms
Calculating -------------------------------------
cached - known ability - pl-benchmark-ips
                        477.663k (± 3.2%) i/s -      2.407M in   5.045721s
cached - unknown ability - pl-benchmark-ips
                        433.737k (± 5.8%) i/s -      2.199M in   5.086746s
uncached - known ability - pl-benchmark-ips
                         12.916k (± 2.1%) i/s -     64.944k in   5.030534s
uncached - unknown ability - pl-benchmark-ips
                         92.715k (± 1.7%) i/s -    465.124k in   5.018246s

Comparison:
cached - known ability - pl-benchmark-ips:   477663.2 i/s
cached - unknown ability - pl-benchmark-ips:   433737.1 i/s - 1.10x  slower
uncached - unknown ability - pl-benchmark-ips:    92715.1 i/s - 5.15x  slower
uncached - known ability - pl-benchmark-ips:    12916.3 i/s - 36.98x  slower

Related Issues

n/a

Suggested version bump

  • Major (backwards incompatible changes)
  • Minor (backwards compatible changes)
  • Patch (API compatible changes)

Checklist

  • Tests have been added or updated to cover any changes in behavior
  • This does not change the API to consume this library, or a suggested version bump has been provided
  • The CHANGELOG.md has been updated
  • No new runtime dependencies have been introduced
Edited by Peter Leitzen

Merge request reports