MVC: Enable Diff-Based Scanning in MRs for Faster GLAS Execution
<!--Implementation issues are used break-up a large piece of work into small, discrete tasks that can move independently through the build workflow steps. They're typically used to populate a Feature Epic. Once created, an implementation issue is usually refined in order to populate and review the implementation plan and weight. Example workflow: https://about.gitlab.com/handbook/engineering/development/threat-management/planning/diagram.html#plan--> ## Why are we doing this work This epic tracks the work needed to release the MVC for a faster [GLAS](https://docs.gitlab.com/user/application_security/sast/gitlab_advanced_sast/) through diff-based scanning in MRs. ## Relevant links https://gitlab.com/groups/gitlab-org/-/epics/16790+ Spike issue documenting the research and decisions that informed the implementation plan: https://gitlab.com/gitlab-org/gitlab/-/issues/536864+ ## Non-functional requirements <!--Add details for required items and delete others.--> - [ ] Documentation: - [ ] Feature flag: - [ ] Performance: - [ ] Testing: ## Implementation plan 1. Partial differential scan in GLAS - Note: These issues are intended to be worked on sequentially, as some depends on the completion of the previous - https://gitlab.com/gitlab-org/gitlab/-/issues/540584+ * Adds the `sast_partial_scan` field to the report module * <span dir="">Add a </span>`partialScan`<span dir=""> param that </span>`gitlab-advanced-sast`<span dir=""> can use to select the type of scan to start</span> - https://gitlab.com/gitlab-org/gitlab/-/issues/540585+ * Add the `sast_partial_scan` field to the security report schema that is used to verify SAST reports in the rails monolith - https://gitlab.com/gitlab-org/gitlab/-/issues/540581+ * Update `lightz-aio` to accept the `selective_scan_changed_files_list_path` flag from `gitlab-advanced-sast` which points to the file containing the diff files, and pass it on to `lightz` - https://gitlab.com/gitlab-org/gitlab/-/issues/540580+ * Decide if full scan or diff scan should be run based on various conditions * Calculate diff files * If diff-scan conditions are met * Configure `lightz-aio` to perform diff-scan * Format SAST report where `sast_partial_scan` is set to `differential` - https://gitlab.com/gitlab-org/gitlab/-/issues/540667+ - https://gitlab.com/gitlab-org/gitlab/-/issues/540670+ 2. Rails backend - https://gitlab.com/gitlab-org/gitlab/-/issues/543636+ * Parse the `gl-sast-report.json` and if it's a diff-based scan, persist the record in the database. - https://gitlab.com/gitlab-org/gitlab/-/issues/543637+ * If the scan is a diff-based scan, it will not determine fixed vulns and only return new and existing vulnerabilities 3. Rails frontend - https://gitlab.com/gitlab-org/gitlab/-/issues/543638+ * MR security widget will only display new and existing vulns for diff-based scans. * It will also indicate that the scan is a partial scan with a link to the documentation on what that means - https://gitlab.com/gitlab-org/gitlab/-/issues/543639+ * Pipeline security tab displays that it's a partial scan with a link to the documentation on what that means 4. Update docs - https://gitlab.com/gitlab-org/gitlab/-/issues/543640+ ## Verification steps <!--Add verification steps to help GitLab team members test the implementation. This is particularly useful during the MR review and the gitlab~11090870 step. You may not know exactly what the verification steps should be during issue refinement, so you can always come back later to add them. 1. Check-out the corresponding branch 1. ... 1. Profit!-->
epic