feat(chunking): add chunking benchmark
What does this MR do and why?
We need benchmarks in place in order to guide other changes. So, while I do implement a performance improvement here, the main objective is to get the benchmark itself in place.
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
You can run the benchmarks using:
make bench
You can capture the output for benchstat:
make bench TEST_OPTIONS="-run=NONE -benchmem -count=10" > baseline.txt
# ...
# make your performance improvements
# ...
make bench TEST_OPTIONS="-run=NONE -benchmem -count=10" > new.txt
Then benchstat will give you the stats:
go run golang.org/x/perf/cmd/benchstat@latest baseline.txt new.txt