Incremental SAST scanning (only scan code changed in the MR)

Customer need

Customer need: faster feedback on existing pipeline-based SAST scans.

Use cases include any large repo, but especially monorepos.

Note: This issue is about SAST, not Secret Detection. On MRs and non-default branches, Secret Detection automatically scans commit-by-commit.

Approach

In merge requests, scan only changed code (and code that is relevant to that code, such as other code that the changed code calls into).

Motivations

Motivations include:

  • Getting results faster.
  • Spending fewer CI/CD minutes on scans.

One motivation that is mentioned but is not on-point: People sometimes also request this as a way to remove remove irrelevant-seeming results from the merge request widget. However, that these types of MR behaviors are likely related to how security reports are handled, which is a separate initiative. (Team members can view related content.) A proper incremental scan would not necessarily cause such results to be omitted from scan output, due to details of how scan reports are processed.

Framing for feature development

This may not be the precise approach we take forever, and may have known limitations on how it applies. It is an optimization.

Limitations may include:

  • Only save cache for default branch
    • Assumption: other branches or branching strategies will still work, but could be slower. (That is, the optimization would be less effective but not incorrect.)
  • In rare cases the incremental scan won't find an issue that would be found by a full scan

Must:

  • Include system limits for stability (to be decided by eng., but can include size or time taken to load)
  • Support self-managed + offline, i. e. not use Cloud Connector for this feature
  • Know the cases in which an incremental scan will miss something, & document this (if any)
  • Not report anything in incremental scan that will not appear in a full/non-incremental scan

Should:/could:

  • Consider starting out only doing this on non-default branches if we have any concern about an incremental scan being incomplete. (This would be to reduce risk.)
  • Enable in 'latest' template w/ variable for early opt-in/testing on stable, before promoting to 100% of traffic.

Other ways to address this

  • We make scans fast (in many cases, an order of magnitude faster than other solutions). This means that the whole-repo scan can, many times, be fast enough for MR review times.
  • If the problem is _scan delay) for users rather than scan runtime specifically, changing the SAST jobs' build stage can help. By default, SAST scans run in the test stage, which doesn't begin until build is over. You can change the stage to an earlier stage in your pipeline so that results are available faster.
    • Note: We need to document better that you can override the stage; this option is available in the UI but not documented. See example MR.
Edited by Connor Gilbert