Update Security Product QA DS expectations with disclosure date filtering
Problem to solve
We update all ~Secure product QA expectations on a regular basis to recognize new vulnerabilities. This can be time consuming and often involves a "broken" build due to new vulnerabilities becoming recognized. As a possible solution to this we should consider filtering vulnerabilities by disclosure date, allowing us a stable list of vulnerabilities scoped to a specific timeframe.
Intended users
Proposal
Update QA expectation diffing (example) to filter vulnerabilities by disclosure date.
Ideally we would leverage the disclosure date, but currently there's no such field in the Dependency Scanning reports, so instead we'll have to leverage the CVE ids. The advisories having a CVE id are compared to a maximum value, like CVE-2019-1234, and filtered out if there were published after this particular id. The maximum value is exposed as DS_QA_MAX_CVE_ID
, and it defaults to the last known CVE id. This ensures that there's no need to update the test projects when changing the QA job; there's no need to update the qa/expect/gl-dependency-scanning.json
files.
Testing
Testing should be more stable with less frequent need to update product or analyzer expectations
What does success look like, and how can we measure that?
Testing should be more stable with less frequent need to update product or analyzer expectations
What is the type of buyer?
Links / references
- ~"dependency scanning" diffing logic: https://gitlab.com/gitlab-org/security-products/ci-templates/blob/c2355cbe5501f993baf61c097f1df39bf4f015f0/includes-dev/qa-dependency_scanning.yml#L8