@@ -56,7 +56,7 @@ When conducting experiments manually (with or without Nancy) it is important to
...
@@ -56,7 +56,7 @@ When conducting experiments manually (with or without Nancy) it is important to
# Main principles of collecting and analyzing results
# Main principles of collecting and analyzing results
1.**Determine bottlenecks.** Were we CPU- or IO-bound? Did you have significant steal time?
1.**Determine bottlenecks.** Were we CPU- or IO-bound? Have we noticed significant steal time?
1. Collect **statistically meaningful data**:
1. Collect **statistically meaningful data**:
1. do not change environment when comparing: if baseline run is to be compared with some changed (with some "delta") ones, always do it **on the same machine**. Example: we need to compare behavior under two conditions: `max_wal_size='1GB'` and `max_wal_size='64GB'`. This should be done on a single machine, in the same environment.
1. do not change environment when comparing: if baseline run is to be compared with some changed (with some "delta") ones, always do it **on the same machine**. Example: we need to compare behavior under two conditions: `max_wal_size='1GB'` and `max_wal_size='64GB'`. This should be done on a single machine, in the same environment.