Skip to content

Make redaction metrics more accurate

Michael Trainor requested to merge regex-metrics into main

Redactions metric was only being incremented for complete redactions, not for regex search/replace . This change will compare the input and output strings from the regex processing, and increment the metric if the strings are different.

I've confirmed that with varying file sizes, the performance has not been negatively impacted as a result. The number of redactions has increased, which is a good sign. The output files are exactly the same, before and after the change.

Below is the comparison of execution time and number of redactions for a 20MB HAR file.

Before:

tmike@Michaels-MacBook-Pro ~/.../support/toolbox/harcleaner main
$ time go run harcleaner -debug -input /Users/tmike/Downloads/cats.har -output /Users/tmike/Downloads/cats-metrics-old.har
INFO[0000] Reading input file: /Users/tmike/Downloads/cats.har
INFO[0000] Parsing and validating input data
INFO[0000] Cleaning HAR data
DEBU[0002] Metrics: {Entries:442 Strings:34814 RegexComparisons:127140 Redactions:845}
INFO[0002] Converting processed data to JSON
DEBU[0002] Creating output file: /Users/tmike/Downloads/cats-metrics-old.har
DEBU[0002] Writing data to output file: /Users/tmike/Downloads/cats-metrics-old.har
INFO[0002] 7743740 bytes written to output file: /Users/tmike/Downloads/cats-metrics-old.har

real	0m2.684s
user	0m2.600s
sys	0m0.248s

After:

tmike@Michaels-MacBook-Pro ~/.../support/toolbox/harcleaner regex-metrics
$ time go run harcleaner -debug -input /Users/tmike/Downloads/cats.har -output /Users/tmike/Downloads/cats-metrics-new.har
INFO[0000] Reading input file: /Users/tmike/Downloads/cats.har
INFO[0000] Parsing and validating input data
INFO[0000] Cleaning HAR data
DEBU[0002] Metrics: {Entries:442 Strings:34814 RegexComparisons:127140 Redactions:884}
INFO[0002] Converting processed data to JSON
DEBU[0002] Creating output file: /Users/tmike/Downloads/cats-metrics-new.har
DEBU[0002] Writing data to output file: /Users/tmike/Downloads/cats-metrics-new.har
INFO[0002] 7743740 bytes written to output file: /Users/tmike/Downloads/cats-metrics-new.har

real	0m2.677s
user	0m2.605s
sys	0m0.236s

Merge request reports