Coverage-guided fuzzing usage ping
Problem to solve
Understanding how our users use the product helps us make better, data-driven decisions about what to do in future iterations. This is difficult without usage ping information to show how users actually use the product.
Proposal
Add usage ping information to report on:
- Number of jobs with fuzz testing run
- Number of fuzz testing jobs that found a faults
- Number of fuzz testing jobs that found no faults
Create a report to report (these two are both to help us address our North Star metric for fuzzing):
- The total number of fuzz results that have had any interaction
- The total number of fuzz results that have been reported
Note: Report should be created using our existing telemetry & reporting systems, most likely on SiSense. This is not intended to be a report that is customer-facing. It can be more "raw" formatted data/graphs.
With each report, provide a way to identify which organization and user in it caused the event to occur. Note: We explicitly do NOT want to collect exact names of organizations or individual users. We are interested in being able to understand if there are a few or many unique users generating reports, but we do not need nor want names of individual users. Consider what anonymization techniques we use in other parts of usage ping and/or using a one-way hash function.