Match SBOM components against malware advisories

What does this MR do and why?

This is the CI half of the malicious-advisory integration: the path where a dependency scanning job posts SBOM components and gets findings back.

Gitlab::VulnerabilityScanning::SecurityReportBuilder matched components only against PackageMetadata::AffectedPackage, so a pipeline never surfaced malware findings even though the advisory tables, models and matching scopes were already on master. The builder now also matches against PackageMetadata::MalwareAffectedPackage and adds the resulting findings to the same dependency scanning report.

One builder serves both entry points, so both gain the behaviour:

• Security::VulnerabilityScanning::ProcessSbomScanService — the sbom_scans API called from CI dependency scanning jobs.
• Ci::JobArtifact#build_security_report — the SBOM-only job path, which generates the report during ingestion.

Gated behind the sbom_scan_malware_findings WIP flag, off by default.

Implementation notes

Findings are additive, not deduplicated. A package covered by both a public and a malware advisory keeps a finding from each. That follows the decision recorded on #612091's sibling issue #594791 (closed), and it matches how the platform already behaves: the vulnerability UUID keys on the primary identifier fingerprint, so two advisories describing the same package already produce two findings today.

Shared traversal. The public path's nested iteration was six levels deep. Rather than duplicate it, each_matching_package now yields (affected_package, occurrence) pairs and both paths use it. add_finding_to_report holds the shared "build it, add it, register its identifiers" step.

The only behavioural difference between the two paths is that malware advisories pass distro: nil, because they are ecosystem-level and never OS-distro specific. affected_occurrence? therefore takes the advisory and distro: as arguments rather than reading them off the affected package, which lets both record types share it.

The malware relation uses the same scope pair as the public one — for_occurrences(...).with_advisory — so batching, the CTE join and eager loading behave identically.

Test coverage

Added to security_report_builder_spec.rb, all inside the existing dependency scanning context:

• A component flagged by a malware advisory produces exactly one malware finding, at critical severity.
• The public advisory findings are kept alongside it, confirming the additive behaviour.
• The GLAM- identifier is registered on the report, not just on the finding. This matters downstream, because malware status is derived from identifier prefixes.
• With a second malware advisory whose affected_range does not cover the component version, only the matching advisory produces a finding — so version filtering is exercised, not just name matching.
• With the feature flag disabled, no malware finding is added and the public advisory findings are unaffected.

Local runs:

• security_report_builder_spec.rb — 18 examples, 0 failures.
• process_sbom_scan_service_spec.rb and ee/spec/models/ee/ci/job_artifact_spec.rb — 162 examples, 0 failures, covering both entry points.
• spec/lib/feature_spec.rb feature flag definitions — 166 examples, 0 failures.
• RuboCop clean.

Database

No schema change and no new index. One new query: the malware branch of SecurityReportBuilder calls

::PackageMetadata::MalwareAffectedPackage.for_occurrences(occurrence_batch).with_advisory

then iterates it with each_batch. It is structurally identical to the public-advisory query already in this method — same for_occurrences CTE shape, same with_advisory preload, same batching — just against pm_malware_affected_packages instead of pm_affected_packages.

The join key is covered exactly:

CREATE INDEX i_pm_malware_affected_packages_on_purl_type_and_package_name
  ON pm_malware_affected_packages USING btree (purl_type, package_name);

for_occurrences joins on (purl_type, package_name), which is that index's full column list. Cost therefore scales with the number of SBOM components in the batch, not with the size of the advisory table: one index lookup per component.

Queries and query plans

Captured from a real request by subscribing to sql.active_record, not reconstructed by hand. occurrences_cte is a VALUES list with one row per SBOM component in the batch, bounded by Security::IngestionConstants::COMPONENTS_BATCH_SIZE. The examples below use 10 components; substitute a realistic batch width for your run.

1. Batch fetch — the main query

EXPLAIN (ANALYZE, BUFFERS)
WITH occurrences_cte(purl_type, name) AS (
  VALUES (6,'base65-85x'),(6,'chalk'),(6,'db-convertor'),(6,'debug'),(6,'express'),
         (6,'lodash'),(6,'ms'),(6,'polymarket-risk-manager'),(6,'semver'),(6,'vitest-agent')
)
SELECT DISTINCT pm_malware_affected_packages.* FROM pm_malware_affected_packages
INNER JOIN occurrences_cte
  ON occurrences_cte.purl_type = pm_malware_affected_packages.purl_type
 AND occurrences_cte.name = pm_malware_affected_packages.package_name
WHERE pm_malware_affected_packages.id >= 1;

postgres.ai plan: https://console.postgres.ai/gitlab/gitlab-production-sec/sessions/55213/commands/158733

2. each_batch boundary probe — runs once per batch

EXPLAIN (ANALYZE, BUFFERS)
WITH occurrences_cte(purl_type, name) AS (
  VALUES (6,'base65-85x'),(6,'chalk'),(6,'db-convertor'),(6,'debug'),(6,'express'),
         (6,'lodash'),(6,'ms'),(6,'polymarket-risk-manager'),(6,'semver'),(6,'vitest-agent')
)
SELECT DISTINCT pm_malware_affected_packages.id FROM pm_malware_affected_packages
INNER JOIN occurrences_cte
  ON occurrences_cte.purl_type = pm_malware_affected_packages.purl_type
 AND occurrences_cte.name = pm_malware_affected_packages.package_name
ORDER BY pm_malware_affected_packages.id ASC LIMIT 1;

postgres.ai plan: https://console.postgres.ai/gitlab/projects/gitlab-production-sec/sessions/55213/commands/158734

3. Advisory preload from with_advisory

Rails emits a flat IN list of the advisory ids matched by one each_batch slice, so substitute real ids from your snapshot. The list holds at most 1000 ids — each_batch is called without of:, so it takes the default — but 50 is the reachable cap, see the note below:

EXPLAIN (ANALYZE, BUFFERS)
SELECT * FROM pm_malware_advisories WHERE id IN (…);

Covered by the primary key index.

postgres.ai plan: https://console.postgres.ai/gitlab/projects/gitlab-production-sec/sessions/55213/commands/158735

Local plans, for reference

Run against a GDK holding 226,551 rows in pm_malware_affected_packages, synced from the staging PDS. Not a substitute for Database Lab, but the plan shape is the point.

Batch fetch — the join is fully index-driven:

Unique  (cost=24.66..24.68 rows=1) (actual time=0.198..0.200 rows=6)
  Buffers: shared hit=51
  ->  Sort  (Sort Method: quicksort  Memory: 25kB)
      ->  Nested Loop  (actual time=0.049..0.183 rows=6 loops=1)
            Buffers: shared hit=39
            ->  Values Scan on "*VALUES*"  (actual rows=10 loops=1)
            ->  Index Scan using i_pm_malware_affected_packages_on_purl_type_and_package_name
                  Index Cond: ((purl_type = "*VALUES*".column1) AND (package_name = "*VALUES*".column2))
                  (actual time=0.017..0.017 rows=1 loops=10)

Ten loops, one per component, each a single index lookup on both columns. 0.2 ms and 51 buffer hits over a 226k-row table.

Boundary probe: Limit → Unique → Sort → Nested Loop, same Values Scan plus index scan, shared hit=42, 0.17 ms.

Advisory preload: Index Scan using pm_malware_advisories_pkey, 6 rows, shared hit=24, 0.12 ms.

Notes for the reviewer

• The DISTINCT comes from with_advisory, which is includes(:malware_advisory).distinct. On SELECT * that sorts by every column, visible as the wide Sort Key above. It sorts matched rows, not table rows — 6 here — so it is bounded by how many of the project's components are actually flagged, which is normally zero or a handful.
• Worst case is a project whose every component is flagged. Then the sort input equals the batch width rather than 6. Still bounded by COMPONENTS_BATCH_SIZE, not by the advisory table.
• The preload IN list is capped at 50 in practice, 1000 structurally. Matched rows come from a CTE of at most COMPONENTS_BATCH_SIZE (50) components, and on the current staging dataset every (purl_type, package_name) is flagged by exactly one advisory — max fan-out 1 across all 226,551 rows. So there is one batch and the IN list is ≤ 50; the pipeline run in Local testing produced 6. Higher fan-out is allowed by the unique index (pm_malware_advisory_id, purl_type, package_name), and reaching the 1000 cap would need average fan-out ≥ 20 across a full batch. If GLAM ever does that, each_batch splits it rather than growing the IN list.
• This query runs only when the feature flag is on, so the pre-rollout cost is unchanged.
• No index is proposed. (purl_type, package_name) already covers the join exactly, and it exists because the ingestion upsert needs it.

Local testing

Steps, and before/after observations from a real pipeline

Verified end to end on a GDK by running an actual dependency scanning pipeline and inspecting the report the analyzer received back, rather than by calling the builder directly.

Setup

1. Import the fixture project. It carries a real package-lock.json pinning malicious npm packages at their flagged versions, plus clean controls:

https://gitlab.com/gitlab-org/govern/threat-insights-demos/verification-projects/bala-test-group/malware-sbom-verification

2. Populate malware advisories. The malware advisory sync fills pm_malware_advisories and pm_malware_affected_packages; it needs sync_malware_advisories and ingest_malware_advisories enabled. Confirm the fixture's components actually match before scanning:

p = Project.find_by_full_path('<group>/malware-sbom-verification')
Sbom::Occurrence.where(project_id: p.id).includes(:component, :component_version).each do |o|
  hits = PackageMetadata::MalwareAffectedPackage
    .for_occurrences([Hashie::Mash.new(name: o.component.name, purl_type: 'npm')]).with_advisory
  puts "#{o.component.name}@#{o.component_version.version} advisories=#{hits.size}" if hits.any?
end

3. Enable the flag. It is wip and off by default:

Feature.enable(:sbom_scan_malware_findings)

4. Clear cached scan results before each run. This step is not optional — see the caveat below.

Security::VulnerabilityScanning::SbomScan.where(project: p).destroy_all

5. Run a pipeline and read the report the analyzer got back, not the project's cumulative vulnerabilities — those persist once created, so they cannot show a per-run difference:

pl = Ci::Pipeline.find(<id>)
art = pl.builds.flat_map(&:job_artifacts).find { |a| a.file_type == 'dependency_scanning' }
art.each_blob { |b| @json = Gitlab::Json.parse(b) }
v = @json['vulnerabilities']
glam = v.select { |x| x['identifiers'].any? { |i| i['name'].to_s.start_with?('GLAM-', 'MAL-') } }
puts "findings=#{v.size} glam=#{glam.size} severities=#{glam.map { |x| x['severity'] }.tally}"

Observations

Same branch, same commit, same fixture — only the feature flag differs.

	Flag off	Flag on
Findings in the report	6	12
GLAM/MAL identified	0	6
Pre-existing GLAD findings	6	6
Severity of the malware findings	—	all Critical

The six malware findings, with the identifier that marks each as malware:

Package	Version	Identifier
base65-85x	5.0.1	MAL-2026-6704
chalk	5.6.1	GLAM-2025-09-01100
db-convertor	1.2.0	GLAM-2026-07-00024
debug	4.4.2	MAL-2025-46974
polymarket-risk-manager	3.5.2	MAL-2026-6712
vitest-agent	0.3.1	MAL-2026-6710

Four things this confirms beyond "findings appear":

• Version filtering, not just name matching. The fixture has chalk twice: 5.6.1 at the root and 5.6.1's clean sibling 5.3.0 nested under express. Only 5.6.1 produced a finding. 5.3.0 matches an advisory by name and is correctly excluded by the range check.
• Severity is Critical, not unknown. That is the value object work in the parent MR reaching the report through the full stack. Contrast the GLAD rows in the same report: GLAD's own malware advisory for chalk lands at unknown, because it has no CVSS and nothing asserts a severity.
• Findings are additive. chalk@5.6.1 and debug@4.4.2 each end up with both a GLAM finding and a GLAD finding, which is the decision recorded on #594791 (closed).
• The flag gates cleanly. With it off, the report is byte-for-byte the pre-existing six findings.

Caveat that will otherwise waste your time

The first run returned zero malware findings on a green pipeline. SbomScanResultCachingService had served a cached result: the SbomScan record for that build pointed at a result_id from six days earlier, so SecurityReportBuilder never ran at all.

Two causes stacked:

• find_cached_scan does not apply the not_expired scope, so a scan well past SbomScan::EXPIRED_AGE (2 days) is still reused when DestroyExpiredSbomScansWorker has not pruned it.
• Separately, a malware-only advisory sync does not invalidate the cache at all — that is !250273 (merged).

So clear the cached scans before each run, or you will get a clean-looking false negative.

Related

• Closes: #612091
• Parent epic: &21156
• Feature flag rollout: #612093
• Overlap decision this relies on: #594791 (closed)
• Proof of concept: !226519 (closed)

Two things a reviewer should know

This MR is stacked. It depends on Advisory.from_malware_advisory, added in !249736 (merged). Target that MR's branch or merge it first.

One follow-up is required before this flag is enabled anywhere. Security::VulnerabilityScanning::SbomScanResultCachingService#advisories_synced_since? checks only PackageMetadata::Checkpoint.for_advisories, so a malware-only sync will not invalidate a cached scan result and the new findings would be silently missing. That fix is #612092 (closed) and it must land before rollout, otherwise staging verification is unreliable.