Scan SBOM occurrences (CVS) against ingested malware advisories

What does this MR do and why?

Continuous vulnerability scanning had no malware path at all. Nothing subscribed to the malware ingestion event, so an ingested malware advisory never produced a vulnerability no matter what else was in place.

This adds the missing trio, mirroring the public advisory path:

Class	Role
PackageMetadata::GlobalMalwareAdvisoryScanWorker	Subscribes to the ingestion event, loads the advisory, delegates
PackageMetadata::MalwareAdvisoryScanService	Thin entry point, matching AdvisoryScanService
Gitlab::VulnerabilityScanning::MalwareAdvisoryScanner	Finds affected SBOM occurrences and creates vulnerabilities

Resolves #612094 (closed)

Stacked on !251402 (merged), which publishes the event this worker subscribes to. Review that one first; this MR's diff will shrink to its own commit once it merges.

How this differs from the public scanner

Every difference follows from malware advisories being ecosystem-level rather than OS-distro specific.

distro is always nil, and container purl types are skipped before the finder runs. occurrence_is_affected? reads distro only on the container-scanning branch, selected by purl type, and the production catalogue contains no container purl types at all. That branch is guarded rather than left unreachable, because unreachable-by-data is not the same as safe: MalwareAffectedPackage uses the full Enums::Sbom.purl_types enum, so apk/rpm/deb/cbl-mariner/wolfi are storable, and name_and_version_for(nil) calls nil.rpartition — a NoMethodError that the SemverDialects::Error rescue does not catch. A container-typed advisory would crash the worker rather than be skipped.

Only the dependency scanning CVS setting applies. The public scanner picks between cvs_for_container_scanning_enabled and cvs_for_dependency_scanning_enabled based on purl type. There is no container-scanning malware case, so only the dependency setting is consulted.

One value object per advisory, not per affected package. Advisory.from_malware_advisory reads nothing from the affected package, so it is built once and memoized rather than rebuilt in the package loop.

No affected_packages filtering. The public scanner filters container-scanning advisories down to supported purl types. Nothing to filter here.

What is deliberately not in this MR

Per-occurrence-ref fan-out. The public scanner branches on Security::VAC.enabled? and builds one finding map per tracked ref. This scanner does not. VAC is not scoped, so it resolves each occurrence's default ref and skips the rest; fan-out is tracked in #612168.

This is deliberately not the same as skipping VAC-enabled projects. vulnerabilities_across_contexts is type: beta, default_enabled: true, so skipping those projects would disable the scanner more or less everywhere. Findings are still produced for every project; only non-default refs are dropped.

The pipeline and the tracked context are both taken from that one ref. They used to come from different places — the pipeline from the occurrence row, the context from the project default — and since one sbom_occurrences row is shared across refs and carries the pipeline of whichever ref last wrote it, the two could disagree. Raised in review by @subashis and @ghavenga.

The checkpoint reset migration. #612097 must ship in a release after this scanner is enabled, so it is intentionally absent. Merging it alongside would reset the sync cursor while scanning was still flag-gated and burn the /all snapshot with no consumer — the mistake already made once in %19.3.

Feature flag

cvs_malware_advisories, wip, default off. Rollout tracked in #612098.

The cvs_ prefix ties the flag to the CVS engine it gates, keeping it distinct from sync_malware_advisories and ingest_malware_advisories, which gate the two earlier stages.

It is checked with :instance as the actor. There is no project or namespace to gate on — the worker runs off a global ingestion event — but passing no actor at all would be worse than it looks: without one, the only usable gradual rollout is percentage_of_time, which Flipper re-rolls on every call. CVS has no re-scan path, so an advisory whose event lost the roll would never be scanned again. :instance makes the rollout all-or-nothing, and satisfies Gitlab/FeatureFlagWithoutActor without a disable comment.

Roll out with --actors, not a plain percentage. When disabled the worker returns before loading the advisory — no query, no scan.

Telemetry

No CVS telemetry, deliberately. The scanner originally reused TrackCvsService, but that service hardcodes EVENT_CATEGORY = 'VulnerabilityScanning::AdvisoryScanner' and EVENT_ACTION = 'global_scan'. Reusing it would have recorded every malware scan as a public advisory scan, with nothing on the event to separate them afterwards — making the existing public numbers wrong rather than merely leaving malware unmeasured. A dedicated event is #624738.

What this MR adds instead is a required scan_type argument on AdvisoryUtils#create_vulnerabilities, surfaced on both the success and failure log lines:

{ "message": "Successfully created vulnerabilities on advisory ingestion",
  "scan_type": "malware_advisory",
  "project_ids_with_upsert": [123, 456] }

There are three callers, and all three now declare themselves:

Caller	scan_type
Gitlab::VulnerabilityScanning::MalwareAdvisoryScanner	:malware_advisory
Gitlab::VulnerabilityScanning::AdvisoryScanner	:advisory
Sbom::CreateVulnerabilitiesService	:advisory

Required rather than defaulted. A default would have kept the two public call sites untouched, which is a diff-size argument rather than a correctness one. The failure mode it creates is worse than the problem it avoids: a caller that forgets the argument is labelled as the public path, so the log actively lies instead of merely omitting a field — and a wrong discriminator is harder to notice than a missing one. Sbom::CreateVulnerabilitiesService demonstrated it, silently inheriting :advisory from the default with nothing recording that anyone had decided that was correct for it. It is correct, and now it says so.

Sbom::CreateVulnerabilitiesService and AdvisoryScanner are distinct entry points that currently share :advisory, so the field cannot separate them. That was left alone rather than inventing a taxonomy for a path this MR does not touch; with the argument required, splitting them later is a one-line change at one call site.

What this field does not cover

Two limits worth stating rather than leaving a reviewer to find:

• The SBOM/CI malware path is not tagged. SecurityReportBuilder (#612091 (closed)) matches malware advisories during a pipeline, but it calls report.add_finding and never create_vulnerabilities, so its findings reach the generic ingestion pipeline without passing this log line. scan_type therefore covers the CVS malware path only, not "all malware vulnerability creation".
• The success line is debug. Gitlab::Logger.log_level reads GITLAB_LOG_LEVEL with a DEBUG fallback, so it is emitted by default — but an environment that raises that to INFO keeps only the failure line. Worth confirming for GitLab.com before leaning on it during the rollout; if it is suppressed there, the case for #624738 is stronger than stated above.

Note that "advisory ingestion" in the message text is pre-existing wording and does not refer to package-metadata ingestion; the line fires on vulnerability creation during a scan. Malware advisory ingestion proper logs separately from MalwareAdvisoryIngestionService, with event: malware_advisory_ingestion and a phase of completed or skipped, and is untouched here.

Making the argument required changed the signature the existing specs call, so advisory_utils_spec.rb and advisory_scanner_spec.rb were both updated.

Test coverage

Spec	Cases
malware_advisory_scanner_spec.rb	Creates a vulnerability for an affected project; asserts critical severity; leaves unaffected projects alone; version outside range creates nothing; unparseable range is skipped rather than raising; cvs_for_dependency_scanning_enabled: false skips the project; a project with no security setting row is still scanned; an advisory with two affected packages yields one vulnerability each; withdrawn advisory creates nothing; advisory with no affected packages; the log carries scan_type: :malware_advisory; with VAC enabled and two tracked refs, exactly one finding is built and it comes from the default ref; a project tracked only on a non-default ref gets nothing; an exact-version disjunction range (=4.0.0 || =5.6.1 || =7.1.2) matches a listed version and misses an unlisted one; a container purl type never reaches the finder
global_malware_advisory_scan_worker_spec.rb	Subscribes to the event; scans the loaded advisory; logs and skips when the advisory is missing; logs and skips a withdrawn advisory, distinctly from a missing one; does nothing when the flag is off
malware_advisory_scan_service_spec.rb	Delegates to the scanner
advisory_utils_spec.rb	scan_type reaches the log as :malware_advisory on both the success and failure paths; omitting scan_type raises, so the absence of a default cannot regress

Spec and RuboCop output

$ bundle exec rspec ee/spec/lib/gitlab/vulnerability_scanning/malware_advisory_scanner_spec.rb
17 examples, 0 failures

$ bundle exec rspec ee/spec/workers/package_metadata/global_malware_advisory_scan_worker_spec.rb
8 examples, 0 failures

$ bundle exec rspec ee/spec/services/package_metadata/malware_advisory_scan_service_spec.rb
1 example, 0 failures

$ bundle exec rspec ee/spec/lib/gitlab/vulnerability_scanning/advisory_utils_spec.rb \
                    ee/spec/lib/gitlab/vulnerability_scanning/advisory_scanner_spec.rb
58 examples, 0 failures   # shared AdvisoryUtils change, incl. the public path

$ bundle exec rspec ee/spec/services/sbom/create_vulnerabilities_service_spec.rb
42 examples, 0 failures   # third caller of create_vulnerabilities

$ bundle exec rubocop <all changed files>
no offenses detected

Database

No migration, no schema change, no new index — read path only. All nine queries are index scans on indexes that already exist.

Two figures worth having before the fold. 99.4% of malware advisories never touch sbom_occurrences at all — 21 buffers, three index lookups, done. And the single worst advisory in the catalogue reads 196,452 occurrences to create 5 findings.

Plans: sec session 55624 and main session 55626. Buffer counts are the figure to read; timings are indicative only — a thin clone serves cold reads from the OS file cache, so the millisecond numbers overstate production latency and are not comparable between runs.

Catalogue measurements, per-query plans, and worst-case analysis

What the advisory catalogue actually contains

Measured on production	Value
Malware advisories	236,165
Affected packages	236,165 — exactly one per advisory, verified by histogram, no tail
Ecosystem mix	npm 220,155 (93.2%), pypi 11,682, gem 3,512, nuget 777, cargo 19, golang 18, maven 2
Container purl types (apk, rpm, deb, cbl-mariner, wolfi)	zero
npm advisories naming a package present in any SBOM on GitLab.com	1,381 of 220,155 — 0.63%

Two things follow, both measured rather than argued.

distro: nil never reaches the container matcher. Container purl types are skipped before the finder runs. The catalogue contains none today, but the column permits them and the matcher raises on a nil distro rather than returning false — see How this differs from the public scanner above.

99.4% of advisories do no scanning work. With no matching sbom_components row, the finder's package_identity is nil, search_scope returns nil, and execute_in_batches returns before sbom_occurrences is touched. Total cost for those: queries A, B and C — 21 buffers.

Per-advisory query trace

The worst case in the catalogue: advisory 318087 → ansi-styles → component 1019 → 196,452 occurrences. This is one of the packages hit by the September 2025 npm compromise, so the tail is real widely-depended-on packages rather than unused typosquats.

#	Query	DB	Index used	Buffers	Plan
A	Load advisory by id (new)	sec	pm_malware_advisories_pkey	7	159458
B	with_affected_packages preload (new scope)	sec	i_pm_malware_affected_packages_unique_for_upsert	7	159459
C	Component identity lookup	sec	idx_sbom_components_on_name_purl_type_component_type_and_org_id	20	159460
D-i	each_batch first boundary	sec	index_sbom_occurrences_on_component_id_and_id (index only, 0 heap fetches)	8	159461
D-ii	each_batch boundary probe	sec	same (index only, 0 heap fetches)	48	159462
D-iii	The batch itself	sec	same	96	159463
E	occurrence_refs preload	sec	idx_sbom_occurrence_refs_on_sbom_occ_id_and_tracked_context_id	547	159464
F	Tracked context, per distinct project	sec	index_security_project_tracked_contexts_on_project_context	9	159465
G	security_setting preload	main	project_security_settings_pkey	13	159466

Every one is an index scan. B confirms the 1:1 shape directly (rows=1) and confirms the preload rides the leading column of the existing unique upsert index — which is why no new index is needed.

The component, component_version, source and project preloads are omitted from the table: all are WHERE id IN (…) primary-key lookups.

Worst case, in full

• 196,452 occurrences → 1,965 each_batch iterations
• Measured per batch: 691 buffers (D-ii 48 + D-iii 96 + E 547)
• Full scan: ≈1.36M buffer accesses
• Findings actually created: 5, across 5 projects

The gap between 196,452 read and 5 written is the affected_range, pinned to the single compromised version (=6.2.2). The scan examines every occurrence of ansi-styles and the range check rejects all but five. That is the mechanism bounding write load, and it is worth stating because it is not visible from the code.

TrackedContextFinder caches by project id and the scanner memoises the finder for the whole scan, so query F runs 5 times, not 196,452 — once per distinct affected project.

Why OFFSET 100 is safe here

The id space is sparse: the first 100 rows of component 1019 span ids 3,439,398 → 5,582,595, and the table's max id is 7,197,697,480. Against an id-only index that boundary probe would be pathological.

It isn't, because index_sbom_occurrences_on_component_id_and_id leads with component_id, so ids are contiguous within the index. D-ii confirms it: Index Only Scan, Heap Fetches: 0, actual rows=101, 48 buffers. The offset walks 101 index tuples, not 2.1M id values.

The occurrence_refs preload

Sbom::PossiblyAffectedOccurrencesFinder chains with_occurrence_refs_for_advisory_scan, which is query E at 547 of the 691 measured per-batch buffers — the largest single component of a batch.

An earlier revision of this MR did not read occurrence_refs, and this section flagged the preload as unused work. That no longer holds: default_branch_ref reads it to resolve each occurrence's default tracked ref, so the preload is now exactly what the scanner runs on. Since the association is already loaded, the lookup is in-memory and adds no queries.

Cross-database boundary

A–F run against sec. G runs against main (gitlab_main_org), so it can never be joined to the sec-side tables. It is a batched preload rather than a per-project query — cvs_enabled_for_project? reads an already-loaded association, because the finder chains with_project_security_setting. The plan confirms the batching: one index scan with project_id = ANY (…) returning all five rows, not five round trips.

Three plan artefacts worth pre-empting

Query C reports dirtied=14 and WAL records for a SELECT. That is hint-bit setting on first touch of the thin clone, not a write performed by the query.

Query D-iii is estimated at rows=48 against rows=100 actual — the planner underestimates because of the correlation between component_id and the id range. It selects the correct index regardless, and the absence of a Rows Removed by Filter line shows the component_version_id IS NOT NULL filter discarded nothing.

Query G's index is project_security_settings_pkey because project_id is the primary key of that table — a 1:1 settings row per project — so the IN list is a primary-key lookup, not a secondary-index scan.

Local reproduction

Run on a GDK with the malware advisory tables populated. A malware advisory is created for semver, a package present in the fixture project's SBOM that carries no malware finding today, so any new vulnerability is unambiguously from this change.

Script, and before / after observations

# /tmp/repro.rb
XID = 'GLAM-2026-08-90500'
project = Project.find_by_full_path('bala-test-group/malware-sbom-verification')

advisory = PackageMetadata::MalwareAdvisory.create!(
  advisory_xid: XID, source_xid: 'glam',
  title: 'Malware in semver', description: 'Repro advisory',
  published_date: 2.days.ago.to_date,
  identifiers: [{ 'type' => 'glam', 'name' => XID, 'value' => XID }], urls: [])
PackageMetadata::MalwareAffectedPackage.create!(
  malware_advisory: advisory, purl_type: :npm, package_name: 'semver', affected_range: '>=0.0.0')

puts "vulnerabilities before : #{project.vulnerabilities.count}"

if defined?(PackageMetadata::MalwareAdvisoryScanService)
  PackageMetadata::MalwareAdvisoryScanService.execute(advisory.reload)
  puts 'scan service           : ran'
else
  puts 'scan service           : NOT DEFINED on this revision'
end

project.reload
puts "vulnerabilities after  : #{project.vulnerabilities.count}"

Before — on master

target component : semver@7.6.3 (project 27)
vulnerabilities before : 12
scan service           : NOT DEFINED on this revision
vulnerabilities after  : 12  (delta 0)
new vulnerability      : none attributable to GLAM-2026-08-90500

The advisory and its affected package are stored, and nothing consumes them: there is no scanner to run.

After — on this branch

target component : semver@7.6.3 (project 27)
vulnerabilities before : 12
scan service           : ran
vulnerabilities after  : 13  (delta 1)
new vulnerability      : Malware in semver | severity=critical

One vulnerability, critical as asserted by the malware path rather than derived from CVSS.

The script calls the scan service directly for determinism. The event wiring that reaches it — subscription, advisory lookup, flag gate — is covered by global_malware_advisory_scan_worker_spec.rb rather than repeated here.

Related

• Epic: &21156
• Event this subscribes to: !251402 (merged)
• VAC follow-up: #612168
• Flag rollout: #612098