Skip to content

Add services to create and process SbomScan models

Olivier Gonzalezrequested to merge
ogonzalez-Add-DS-Sbom-Scan-API-47c97932 into master

Overall goal

This is a new approach to our Dependency Scanning feature that reuses the GitLab Sbom Vulnerability Scanner developed in the rails platform for scanning SBOM files generated in a CI job. This allows GitLab to centralize the Dependency Scanning feature on a single scanning engine for various scanning contexts. Please see the epic Bring security scan results back into the Depen... (&17150) for more details.

This MR is part of a stacked diff:

  1. Introduce SbomScan model and Uploader for DS using SBOM (!195058 (merged))
  2. ➡️ Add services to create and process SbomScan models (!195059 (merged))
  3. Add service and worker to destroy expired SbomScan models (!195061 (merged))
  4. Add Sbom Scan API endpoints with direct-upload support (!195062 (merged))

Thus, the complete implementation with all code changes is available in the last MR (!195062 (merged)).

Notes:

  • This feature will be released behind the feature flag dependency_scanning_sbom_scan_api included in the last MR (with the "user facing" API).
  • These SBOM scans are only meant to be ephemeral and thus they don't interact with the Vulnerability Management system.

Video recordings

What does this MR do and why?

This MR introduces the core business logic services. It adds service objects to handle the creation of SbomScan records and the processing workflow that executes vulnerability analysis on uploaded SBOM documents using the GitLab SBOM Vulnerability Scanner.

The implementation includes proper error handling for cases like files being too large, files not being available yet, or scanning failures. It also includes a retry mechanism for temporary failures and uses a background job queue to handle the processing at scale without blocking other operations. Though, we expect these scans to be processed quickly to prevent hanging the CI job for too long. Please see the details in the issue describing the choices for the sidekiq worker attributes like the concurrency_limit.

Note that the vulnerability scanning logic itself is not a new feature, it's been previously used in the Beta stage of the Dependency Scanning Using SBOM feature. This MR reuses the SecurityReportBuilder to perform the analysis and output results in the desired format.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

This MR only contains a subset of the code changes required for the feature. Please use the last MR of the stacked diff (!195062 (merged)) to test the feature.

This particular MR's changes can be tested in isolation by checking out the branch, running the DB migration (if not already done with the first MR of the stacked diff (!195058 (merged)) and running the tests:

git checkout ogonzalez-Add-DS-Sbom-Scan-API-47c97932
bin/rails db:migrate:up:main RAILS_ENV=development --trace
bundle exec rspec ee/spec/lib/api/entities/security/vulnerability_scanning/finding_spec.rb ee/spec/services/security/vulnerability_scanning/create_sbom_scan_service_spec.rb ee/spec/services/security/vulnerability_scanning/process_sbom_scan_service_spec.rb ee/spec/workers/security/vulnerability_scanning/process_sbom_scan_worker_spec.rb spec/workers/every_sidekiq_worker_spec.rb

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Olivier Gonzalez

Merge request reports