Secure and Defend North Star Metrics
This issue captures the North Star metrics for the Secure and Defend section as well as the individual North Star metrics that roll up to it.
Section
Section Name | Product Director | North Star Metric |
---|---|---|
Secure and Defend | @david | Average graded rating of all stages in the section |
Stage
Stage Name | Product Director | North Star Metric |
---|---|---|
Secure | @david | Average graded rating of all categories in the stage |
Defend | @david | Average graded rating of all categories in the stage |
Categories
Category Name | Product Manager | North Star Metric | Collectable today? Or net new work? |
---|---|---|---|
SAST | @tmccaslin | True positive - (.25 * False Negative) using benchmark | Y |
Secret Detection | @tmccaslin | True positive - (.25 * False Negative) using benchmark | N - need secret benchmark |
DAST | @derekferguson | Ratio of findings that have been interacted with and total # of findings | Y |
PKI Management | @derekferguson | Number of certs added for management | No - category not at minimal yet |
Dependency Scanning | @NicoleSchwartz | Ratio of acted on findings* to total findings | N |
Container Scanning | @NicoleSchwartz | Ratio of acted on findings* to total findings | N |
License Compliance | @NicoleSchwartz | Ratio of Licenses Detected with policies [sum(allowed+denied)] to Licenses Detected total | N |
Fuzz Testing | @stkerr | Ratio of number of identified faults that have been interacted with and number of all identified faults. | No, net new |
Web Application Firewall | @sam.white | Number of network packets processed | No, net new, the current metrics are not accurate |
Container Network Security | @sam.white | Number of network packets processed | No, net new |
Container Behavior Analytics | @sam.white | Number of attacks detected | No, category has not reached minimal yet |
Vulnerability Management | @matt_wilson | Number of dashboard views | Maybe, depends on if this is captured in DB currently |
UEBA | @matt_wilson | TBD | No, metric captured will be defined after Problem Validation complete and Solution determined |
- Acted on findings are - create issue, dismiss, fixed, suggested solution. We will EXCLUDE "dismiss-false positive" when we can easily exclude those (right now it's only a text comment field not a separate action).
Scoring Grades
We will score each metric with an A, B, C, D, or F grade. This will mimic what we do for UX scorecards
For metrics that are measured as a ratio, we will use the following scale:
Grade | Score |
---|---|
A | 90-100% |
B | 80-90% |
C | 70-80% |
D | 60-70% |
F | 0-60% |
Any adjustments or changes for each metric will be noted there.
Examples
Dependency Scanning, Container Scanning (for branches and main, but could star with main only?) ( SUM (Fixed+dismissed+create issue+suggested solution) ) / (total finding occurrences)
Edited by Taylor McCaslin