Design: SAST diff scanning
Context & problem to solve
From Faster Advanced SAST: Diff-based scanning in MRs (&16790 - closed):
Scan a target project faster by scanning only changed parts of the codebase.
For this type of faster scan:
- We will focus on the code modified in the MR (essentially the
git diff). - We will not rely on sharing state (such as a full-scan baseline) between runs.
This is a first type of improvement, leading toward a more technically complex system described in future iterations like Incremental scanning for Advanced SAST (skip un... (&15545).
The result of a diff-based scan is, indeed, a "partial report" because it won't contain all the findings for the project since only a subset of the files were scanned.
Reporting (MR widget, pipeline report)
This scan type may produce false negatives in the scanned files, and probably won't know about scan results in unrelated files.
Because of this:
-
We shouldn't make any assertions about an MR having resolved a preexisting vulnerability. This means we should not list "Resolved" findings.
- If a file wasn't scanned, we should ignore vulnerabilities associated with it.
-
We should only highlight new findings. We should compare the diff-based scan's results against the vuln report so that we do not report those findings as though they were new.
- This is existing functionality, so I'm not sure which parts of the UI should be modified. It's a good question whether or not we should make it clear that the results are not from a full scan. I defer to UX to determine if and how. That's how I interpret "highlight" anyway.
- We should consider an in-UI notice that a partial scan was used, and link to documentation explaining the cases in which this can cause FNs.
- We should consider documenting how you can re-run the pipeline or job with a full scan if you really want to verify that you fixed something.
From Connor:
We are planning for this diff-based scanning to be used only on MRs, so this kind of partial scan would not ever happen on a default branch. Hence we would never go update the Vulnerability Report to say "xyz finding is No Longer Detected".