Store security findings detected in SBOMs when pipeline completes

Why are we doing this work

The UI relies on security findings stored in the database:

• pipeline page
• security widget of the MR page, as part of Use security_findings for security MR widget re... (#390185 - closed)

For this UI to work, we need to do the following:

• Store security scans similar to Security::StoreScansServices (plural) and Security::StoreScanServices (singular).
• Store security findings using Security::StoreFindingsService

This applies to any branch. See #395704 (comment 1308790914)

This behavior is enabled by the feature flag that enables Vulnerability Continuous Scans. #395692 (closed)

Detecting vulnerabilities in a pipeline is covered by Match SBOM components to known advisories (#371055 - closed).

Relevant links

Non-functional requirements

• Documentation: #416074 (closed)
• Feature flag: dependency_scanning_using_sbom_reports #395692 (closed)
• Performance:
• Testing: rspec unit tests

Proposal

• Update the Security::StoreScansServices (plural) and StoreScanServices (singular) so that they handle CycloneDX SBOM like Dependency Scanning report.
• Update JobArtifact#security_report so that it generates a Dependency Scanning report in memory, using the scanner implemented in Generate new in-memory Dependency Scanning repo... (#408257 - closed).

Implementation plan

• Update Security::StoreScansService#parse_report_file? (StoreScans, plural) so that it returns true for cyclonedx file types if the :dependency_scanning licensed feature is enabled. ¹
• Update Security::StoreScanService#security_scan (StoreScan, singular) so that it maps the scan_type from cyclonedx to dependency_scanning. See PoC. ²
• Update EE:Ci::JobArtifact#security_report so that it converts an SBOM to a security report using the scanner class implemented in Generate new in-memory Dependency Scanning repo... (#408257 - closed). Decomposing into a separate method is recommended for both the artifact parsing and the SBOM conversion.

Previous implementation plan(s)

Event driven approach with new event, subscriber, and producer

• Add a new service, name TBD
  1. TBD: service arguments
  2. Execute matcher implemented in Match SBOM components to known advisories (#371055 - closed).
  3. Convert matcher output to a report with security findings.
  4. Generate Security::Scan. TBD: where and how?
  5. Call StoreFindingsService with security scan and security findings.
• Add a new event for when the pipeline completes, name TBD
• Add a new subscriber to that event; this acts as a worker

Verification steps

Feature test

To be tested an instance that's been synced up with the advisory database (#417191 (closed)).

• Set up a project compatible with GitLab Dependency Scanning (DS).
• Enable the feature for that project.
• Update the CI config for the default branch.
  • Include the DS CI template
  • Override DS jobs so that they only upload CycloneDX SBOMs, and not the DS reports.
  • This triggers a pipeline for the default branch.
• Check Security tab of the pipeline page.
• Check vulnerability report.
• Check dependency list.
• Create an MR that adds DS vulnerabilities.
• Check security MR widget (diff of DS vulnerabilities).
• Merge the MR.
• Check the new vulnerability report.
• Check the new dependency list.

Non-regression tests

Non-regression test for Container Scanning

• Set up a project compatible with both Container Scanning (CS) and Dependency Scanning (DS).
• Enable the feature for that project.
• Update the CI config for the default branch.
  • Include CI templates for both CS and DS.
  • Override DS jobs so that they only upload CycloneDX SBOMs, and not the DS reports.
  • This triggers a pipeline for the default branch.
• Check Security tab of the pipeline page.
• Check vulnerability report.
• Check dependency list.

Footnotes

1. To handle all the edge cases of the tiered offering difference between container scanning and dependency scanning, we'll need to provide a more complex solution. See #395704 (comment 1485710503) ↩

2. This isn't 100% accurate because it can also be container_scanning. A separate issue will be needed to address this in the container scanning implementation. ↩