Skip to content

Add GlobalAdvisoryScanWorker

What does this MR do and why?

This MR makes Continuous Vulnerability Scanning (CVS) on advisory DB change Globally Available (GA) when the flag global_dependency_scanning_on_advisory_ingestion is enabled.

  • Add new development feature flag named global_dependency_scanning_on_advisory_ingestion (disabled by default).
  • Add new global param to Sbom::PossiblyAffectedOccurrencesFinder, ::GitLab::VulnerabilityScanning::AdvisoryScanner, and PackageMetadata::AdvisoryScanService. When it's true, the scan is global. However, AdvisoryScanner skips global scans when global_dependency_scanning_on_advisory_ingestion is disabled.
  • Add new global param to the TrackCvsService. The event action is global_scan when global is true.
  • Add new GlobalAdvisoryScanWorker. It subscribes to the ::PackageMetadata::IngestedAdvisoryEvent. It sets global to true when calling the AdvisoryScanService.

The AdvisoryScanWorker, which handles projects where CVS has been explicitly enabled in security settings, is updated to set global to false.

Projects where CVS has been enabled (in project settings) are scanned by the AdvisoryScanWorker and the GlobalAdvisoryScanWorker. This is safe b/c the AdvisoryScanner is idempotent; it won't create duplicate vulnerabilities. There's a bit of a overhead but it's negligible b/c CVS has been enabled in very few projects (~50).

Eventually the AdvisoryScanWorker needs to be removed. This will be covered by a separate issue, and implemented in a latter milestone.

The documentation is updated in a separate issue.

See CVS on advisory DB changes (GA) (#426089 - closed)

Query plans

When global is true, the PossiblyAffectedOccurrencesFinder behaves like when it was introduced in !131454 (merged). As a result, we can rely on the query plans that were shared back then; these query plans have already been approved.

PossiblyAffectedOccurrencesFinder was later updated in !128941 (merged) to only consider projects where project_security_settings.continuous_vulnerability_scans_enabled is true. Again, this changes is reverted when global is true.

Historically the finder has been introduced and modified by the following MRs, in that exact order:

  1. Only include sbom_occurrences with CVS enabled (!131454 - merged)
  2. Memoize component_id method (!131211 - merged)
  3. Add PossiblyAffectedOccurrencesFinder class (!128941 - merged)

Queries should leverage the following DB indexes:

CREATE UNIQUE INDEX index_sbom_components_on_component_type_name_and_purl_type ON sbom_components USING btree (name, purl_type, component_type);

CREATE INDEX index_sbom_occurrences_on_component_id_and_id ON sbom_occurrences USING btree (component_id, id);

These haven't changed since Add PossiblyAffectedOccurrencesFinder class (!128941 - merged).

Logging

As suggested during the review, scan workers now log an error when the advisory cannot be found, instead of an info message.

Performance

Right now global scans are too slow to be enabled in production, so we won't enable the new feature flag until Ingest vulnerabilities from multiple projects a... (#420768 - closed) and Make AdvisoryScanner ingest multiple finding maps (#429834 - closed) are completed. The code introduce in this MR shouldn't change though.

See #423578 (comment 1615208412) for context.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #426089 (closed)

Edited by Olivier Gonzalez

Merge request reports