Secure and Defend North Star Metrics

This issue captures the North Star metrics for the Secure and Defend section as well as the individual North Star metrics that roll up to it.

Section

Section Name	Product Director	North Star Metric
Secure and Defend	@david	Average graded rating of all stages in the section

Stage

Stage Name	Product Director	North Star Metric
Secure	@david	Average graded rating of all categories in the stage
Defend	@david	Average graded rating of all categories in the stage

Categories

Category Name	Product Manager	North Star Metric	Collectable today? Or net new work?
SAST	@tmccaslin	True positive - (.25 * False Negative) using benchmark	Y
Secret Detection	@tmccaslin	True positive - (.25 * False Negative) using benchmark	N - need secret benchmark
DAST	@derekferguson	Ratio of findings that have been interacted with and total # of findings	Y
PKI Management	@derekferguson	Number of certs added for management	No - category not at minimal yet
Dependency Scanning	@NicoleSchwartz	Ratio of acted on findings* to total findings	N
Container Scanning	@NicoleSchwartz	Ratio of acted on findings* to total findings	N
License Compliance	@NicoleSchwartz	Ratio of Licenses Detected with policies [sum(allowed+denied)] to Licenses Detected total	N
Fuzz Testing	@stkerr	Ratio of number of identified faults that have been interacted with and number of all identified faults.	No, net new
Web Application Firewall	@sam.white	Number of network packets processed	No, net new, the current metrics are not accurate
Container Network Security	@sam.white	Number of network packets processed	No, net new
Container Behavior Analytics	@sam.white	Number of attacks detected	No, category has not reached minimal yet
Vulnerability Management	@matt_wilson	Number of dashboard views	Maybe, depends on if this is captured in DB currently
UEBA	@matt_wilson	TBD	No, metric captured will be defined after Problem Validation complete and Solution determined

• Acted on findings are - create issue, dismiss, fixed, suggested solution. We will EXCLUDE "dismiss-false positive" when we can easily exclude those (right now it's only a text comment field not a separate action).

Scoring Grades

We will score each metric with an A, B, C, D, or F grade. This will mimic what we do for UX scorecards

For metrics that are measured as a ratio, we will use the following scale:

Grade	Score
A	90-100%
B	80-90%
C	70-80%
D	60-70%
F	0-60%

Any adjustments or changes for each metric will be noted there.

Examples

Dependency Scanning, Container Scanning (for branches and main, but could star with main only?) ( SUM (Fixed+dismissed+create issue+suggested solution) ) / (total finding occurrences)