MVC: Enable Diff-Based Scanning in MRs for Faster GLAS Execution
<!--Implementation issues are used break-up a large piece of work into small, discrete tasks that can
move independently through the build workflow steps. They're typically used to populate a Feature
Epic. Once created, an implementation issue is usually refined in order to populate and review the
implementation plan and weight.
Example workflow: https://about.gitlab.com/handbook/engineering/development/threat-management/planning/diagram.html#plan-->
## Why are we doing this work
This epic tracks the work needed to release the MVC for a faster [GLAS](https://docs.gitlab.com/user/application_security/sast/gitlab_advanced_sast/) through diff-based scanning in MRs.
## Relevant links
https://gitlab.com/groups/gitlab-org/-/epics/16790+
Spike issue documenting the research and decisions that informed the implementation plan: https://gitlab.com/gitlab-org/gitlab/-/issues/536864+
## Non-functional requirements
<!--Add details for required items and delete others.-->
- [ ] Documentation:
- [ ] Feature flag:
- [ ] Performance:
- [ ] Testing:
## Implementation plan
1. Partial differential scan in GLAS
- Note: These issues are intended to be worked on sequentially, as some depends on the completion of the previous
- https://gitlab.com/gitlab-org/gitlab/-/issues/540584+
* Adds the `sast_partial_scan` field to the report module
* <span dir="">Add a </span>`partialScan`<span dir=""> param that </span>`gitlab-advanced-sast`<span dir=""> can use to select the type of scan to start</span>
- https://gitlab.com/gitlab-org/gitlab/-/issues/540585+
* Add the `sast_partial_scan` field to the security report schema that is used to verify SAST reports in the rails monolith
- https://gitlab.com/gitlab-org/gitlab/-/issues/540581+
* Update `lightz-aio` to accept the `selective_scan_changed_files_list_path` flag from `gitlab-advanced-sast` which points to the file containing the diff files, and pass it on to `lightz`
- https://gitlab.com/gitlab-org/gitlab/-/issues/540580+
* Decide if full scan or diff scan should be run based on various conditions
* Calculate diff files
* If diff-scan conditions are met
* Configure `lightz-aio` to perform diff-scan
* Format SAST report where `sast_partial_scan` is set to `differential`
- https://gitlab.com/gitlab-org/gitlab/-/issues/540667+
- https://gitlab.com/gitlab-org/gitlab/-/issues/540670+
2. Rails backend
- https://gitlab.com/gitlab-org/gitlab/-/issues/543636+
* Parse the `gl-sast-report.json` and if it's a diff-based scan, persist the record in the database.
- https://gitlab.com/gitlab-org/gitlab/-/issues/543637+
* If the scan is a diff-based scan, it will not determine fixed vulns and only return new and existing vulnerabilities
3. Rails frontend
- https://gitlab.com/gitlab-org/gitlab/-/issues/543638+
* MR security widget will only display new and existing vulns for diff-based scans.
* It will also indicate that the scan is a partial scan with a link to the documentation on what that means
- https://gitlab.com/gitlab-org/gitlab/-/issues/543639+
* Pipeline security tab displays that it's a partial scan with a link to the documentation on what that means
4. Update docs
- https://gitlab.com/gitlab-org/gitlab/-/issues/543640+
## Verification steps
<!--Add verification steps to help GitLab team members test the implementation. This is particularly useful
during the MR review and the gitlab~11090870 step. You may not know exactly what the
verification steps should be during issue refinement, so you can always come back later to add
them.
1. Check-out the corresponding branch
1. ...
1. Profit!-->
epic