SbomScan result caching - service and api layer

What does this MR do and why?

Currently, SBOM scans are triggered on every pipeline run regardless of whether dependencies have changed, wasting computational resources by processing identical dependencies and generating the same results. This MR introduces a new API endpoint for sbom scanning, adds a new service, and update services that deal with sbom scan results.

These additions enable the client (DS analyzer) to check if an SBOM has been previous processed before uploading and triggering a new resource intensive scan. The client makes a POST to the API endpoint. If the sbom_digest exists in the project and it has no advisories fresher than last scan times, a new SbomScan will be generated corresponding for the current job, but reusing the SbomScanResult of the original.

References

Parent MR: Avoid triggering a re-scan when dependencies ha... (#562694) • Igor Frenkel • 18.7 • Needs attention

This MR is one of 3

1. SbomScan result caching - models and db changes (!213584) • Igor Frenkel • 18.7
2. Current 👉 SbomScan result caching - service and api layer (!213586) • Igor Frenkel • 18.7
3. Pending: observability events for caching

Analyzer MR: Add sbom scan caching support (gitlab-org/security-products/analyzers/dependency-scanning!400) • Igor Frenkel • 18.7

Change summary

• API
  • Add new endpoint at /api/v4/jobs/:job_id/sbom_scans/:sbom_digest
• Services
  • Add SbomScanResultCachingService responsible for finding matching sbom and checking freshness.
  • Update CreateSbomScanService to store sbom_digest for later retrieval.
  • Update ProcesSbomScanService to store scan results in SbomScanResult which can be shared among many SbomScan records.
  • Update DestroySbomScansService to remove orphaned SbomScanResult entries.
• Feature Flag
  • Add dependency_scanning_sbom_scan_result_cache which makes the api endpoint available to the client

Rollout plan

See issue for detail: #562694

Screenshots or screen recordings

Video demonstrating validation steps: screen recording

How to set up and validate locally

• Checkout this MR branch (ifrenkel/562694-sbom-scan-result-caching-2)
• rails db:migrate (migrations are in MR 1 in the MR series)
• Create test project in your gdk
  • gdk rails console to enable feature flag for newly created project: Feature.enable(:dependency_scanning_sbom_scan_result_cache, Project.last)
  • Add ci config (.gitlab-ci.yml) with DS job targeting analyzer image with new functionality (analyzer MR)
  • Add lockfile (Gemfile.lock)
  • Make initial commit - find job, job log has: cache miss
  • Make new commit (unrelated to lock file) - job log has: cache hit
  • Make new commit updating dependency version in the lockfile - job log has: cache miss
  • Make new commit (unrelated to lock file) - job log has: cache hit

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.