Show one finding when GitLab and a third-party tool report the same vulnerability

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Why are we doing this work

The vulnerability report shows one finding when GitLab's own scan and a third-party SBOM report the same vulnerability, in either ingestion order and also for findings that continuous vulnerability scanning creates. The one finding keeps GitLab's identifier, scanner and severity.

A finding is an entry in the vulnerability report, and its primary identifier is the first identifier it lists. An incoming finding is one in the report being ingested, and a stored finding is one already in the vulnerability report.

GitLab's own finding is led by a Gemnasium identifier, the identifier from GitLab's advisory database. Third-party tools lead with a CVE, a public identifier for one vulnerability, or a GHSA, a GitHub Security Advisory identifier, so the same vulnerability becomes two findings. GitLab matches the two when they share an identifier, a CVE or a GHSA. A CycloneDX vulnerability entry has one structured id, and a second identifier appears as a reference, usually a URL.

Update ingestion and continuous vulnerability scanning to:

  • Add an identifier override service that reuses a stored finding that shares an identifier with an incoming finding for the same package and manifest
  • Keep GitLab's primary identifier, scanner and severity on a matched finding, with the third party's identifiers as secondary identifiers
  • Make continuous vulnerability scanning, the background job that creates findings when a new advisory is published, reuse a stored third-party finding instead of creating a second one
  • In the pipeline path, match a third-party finding at the default location, the fixed path 3rd Party Input File Path Unknown that a finding with no recorded path gets, with a finding for the same package at a known path by package alone

Continuous vulnerability scanning does not match a finding at the default location with a finding at a real manifest, which is a documented limit.

This applies when findings from the child issue "Create vulnerability findings from an SBOM's own vulnerability data" exist, and GitLab's own report and the third-party SBOM are uploaded in the same pipeline. A later pipeline's ingestion removes the dependency rows that the reports of an earlier pipeline wrote.

It belongs to epic &14760.

  • Covers requirement 8 in part: a matched finding is stored once and keeps GitLab's values
  • Covers requirement 7 in part: a finding that continuous vulnerability scanning creates for a third-party component is matched to the stored finding
  • Needs first for MR 3: MR 4 of the child issue "Create vulnerability findings from an SBOM's own vulnerability data"

The diagram below shows the path that creates findings from an SBOM's vulnerability entries, which Create vulnerability findings from an SBOM's ow... (#633169) adds. This issue adds the decision "GitLab already has this finding?", which keeps GitLab's finding and adds the SBOM's identifiers when GitLab already has it. Yellow marks what the epic adds.

New path from the SBOM's vulnerability entries

  • Epic &14760
  • Vulns Across Contexts UUID refactor, not a prerequisite because the matched finding keeps GitLab's primary identifier: #587624

Non-functional requirements

  • Documentation: documented by the child issue "Document the supported generators and the CycloneDX fields we read"
  • Feature flag: to be decided
  • Performance: continuous scanning runs at most three queries per batch
  • Testing: specs per case in both ingestion orders, a re-ingestion spec that counts the rows written and a query-count spec for one batch of continuous scanning

Implementation plan

The merge requests are done in the order listed, each based on the one it names.

  • MR 1: Add the identifier override service, which finds a stored finding with a shared identifier for the same package and branch. Based on master.
  • MR 2: Call the service when a scan is stored. Based on MR 1.
  • MR 3: Match one to one at the same manifest. Based on MR 2 and MR 4 of the issue "Create vulnerability findings from an SBOM's own vulnerability data".
  • MR 4: Match a finding at the default location with a finding at a known path for the same package, after known paths. Based on MR 3.
  • MR 5: Put the reuse rule in one class that two later merge requests call. Based on MR 3.
  • MR 6: Let a GitLab finding reuse a stored third-party finding, so the result no longer depends on the ingestion order. Based on MR 5.
  • MR 7: Keep GitLab's primary identifier and scanner on a matched finding, with the third party's identifiers as secondary. Based on MR 6.
  • MR 8: Keep GitLab's severity on a matched finding. Based on MR 7.
  • MR 9: Add the matching service for continuous vulnerability scanning. It uses the class of MR 5. Based on MR 5.
  • MR 10: Call that service when continuous scanning creates findings, without failing the batch on a lookup error. Based on MR 9 and MR 7.

Acceptance Criteria

  1. A third-party finding and GitLab's finding for the same vulnerability, package and manifest appear once, in either ingestion order
  2. A matched finding keeps GitLab's primary identifier, scanner and severity
  3. A matched finding carries the third party's identifiers as secondary identifiers
  4. Findings at different known manifests are never matched with each other
  5. Ingesting the same SBOM again leaves the findings unchanged
  6. Continuous vulnerability scanning reuses a stored third-party finding for the same vulnerability, and creates no second finding

Out of Scope

  • Matching findings that share no identifier
  • Reading identifiers from an entry's references and advisory URLs, which is open in the epic's open question on identifiers read from URLs
  • Removing duplicates that already exist
  • Matching a finding at the default location with a finding at a real manifest in continuous vulnerability scanning, which is a documented limit

Verification steps

  1. Check out the branch of the last merge request listed in the Implementation plan, on a GitLab Development Kit (GDK) instance with a plan that has dependency scanning, container scanning or license scanning (Ultimate).
  2. In a project, run one pipeline whose jobs upload GitLab's own report and the third-party SBOM, with GitLab's report ingested first, and read the finding count, primary identifier, scanner and severity of each finding in the vulnerability report. Repeat in a second project with the third-party SBOM first, then run the pipeline again, and compare the counts and the matched pairs.
  3. Ingest a third-party SBOM, trigger continuous scanning for an advisory that lists an identifier of one of its stored findings by running Gitlab::VulnerabilityScanning::AdvisoryScanner.scan_projects_for(advisory) in a Rails console, and read the vulnerability report.
  4. Compare the readings with the expected values attached to this issue.
Edited by 🤖 GitLab Bot 🤖