Prepare GLAS diff-based scan data for rails frontend

Summary

This issue ensures that for GLAS diff-based scans (indicated by the partial_scan_mode column in the security_scans table, added in #543636 (closed)), backend only passes new and existing vulnerabilities to the frontend.
See the Hide Fixed Vulnerabilities section of this document for the full context.

See MVC: Enable Diff-Based Scanning in MRs for Fast... (&17758 - closed) on how this issue fits in the whole plan.

References

Main diff-based scanning epic: Faster Advanced SAST: Diff-based scanning in MRs (&16790 - closed)

MVC implementation epic: MVC: Enable Diff-Based Scanning in MRs for Fast... (&17758 - closed)

Implementation Plan For MR widget API changes

1. Update Security::FindingsFinder to:
   1. Add a new partial_scan parameter
   2. Modify the lateral_relation to filter by partial_scan to return findings from a partial scan. See example query.
2. Update Ci::CompareSecurityReportsService and check if:
   1. Pipeline has Partial scans and/or Full scans, use Security::FindingsFinder to query for both full scan findings and partial scan findings.
   2. Pipeline only has full scan, only query for full scan findings.
   • Note that both queries will return up to 25 findings.
3. Update Gitlab:Ci::Reports::Security::SecurityFindingsReportsComparer to:
   1. Add a added_partial method.
   2. Exclude partial scan findings from fixed_findings
4. Update Vulnerabilities::FindingReportsComparerEntity to include an added_partial in the api request

MR Security Widget

Add a new added_partial filed to the security report API:

{
  ...
  "added": [] // For full scan findings
  "added_partial": [] // For partial scan findings
  ...
}

frontend issue: Display GLAS diff-based scan in MR security widget (#543638 - closed)

Pipeline Security Tab

1. backend issue: Prepare GLAS diff-based scan data for rails fro... (#554927 - closed) • Ugo Nnanna Okeadu
2. frontend issue: Display GLAS diff-based scan in pipeline securi... (#543639 - closed) • Savas Vedova • 18.5

Verification steps

Existing verification project: https://gitlab.com/gitlab-org/govern/threat-insights-demos/verification-projects/partial-scans

1. Create a new project

2. Enable the vulnerability_partial_scans feature flag on the project

3. Add this .gitlab-ci.yml using a merge request:

   sast:
     stage: test
     script: wget https://gitlab.com/gitlab-org/gitlab/-/raw/5d7bc4e75ae688b968a6debc0b8f62e7eb1f54d1/ee/spec/fixtures/security_reports/master/gl-sast-report-differential.json
     artifacts:
       reports:
         sast: gl-sast-report-differential.json
     rules:
       - if: $CI_COMMIT_BRANCH != $CI_DEFAULT_BRANCH

4. Go to /<project>/-/merge_requests/1/security_reports?type=sast&partial_scan=true twice (the first time enqueues reactive caching). The partial scan results should show.

   Verification project: https://gitlab.com/gitlab-org/govern/threat-insights-demos/verification-projects/partial-scans/-/merge_requests/1/security_reports?type=sast&partial_scan=true

5. Send this graphql query:

   query enabledScans($fullPath: ID!, $pipelineIid: ID!) {
     project(fullPath: $fullPath) {
       id
       pipeline(iid: $pipelineIid) {
         id
         enabledSecurityScans {
           ready
           apiFuzzing
           clusterImageScanning
           containerScanning
           coverageFuzzing
           dast
           dependencyScanning
           sast
           secretDetection
         }
         enabledPartialSecurityScans {
           ready
           apiFuzzing
           clusterImageScanning
           containerScanning
           coverageFuzzing
           dast
           dependencyScanning
           sast
           secretDetection
         }
       }
     }
   }

   variables:

   {
     "fullPath": "gitlab-org/govern/threat-insights-demos/verification-projects/partial-scans",
     "pipelineIid": 3
   }

   It should return the following response:

   {
     "data": {
       "project": {
         "id": "gid://gitlab/Project/74018706",
         "pipeline": {
           "id": "gid://gitlab/Ci::Pipeline/2012021717",
           "enabledSecurityScans": {
             "ready": true,
             "apiFuzzing": false,
             "clusterImageScanning": false,
             "containerScanning": false,
             "coverageFuzzing": false,
             "dast": false,
             "dependencyScanning": false,
             "sast": true,
             "secretDetection": false
           },
           "enabledPartialSecurityScans": {
             "ready": true,
             "apiFuzzing": false,
             "clusterImageScanning": false,
             "containerScanning": false,
             "coverageFuzzing": false,
             "dast": false,
             "dependencyScanning": false,
             "sast": true,
             "secretDetection": false
           }
         }
       }
     },
     "correlationId": "3b31cf8c152ec5b8ad39132838f65953"
   }