Update `vulnerability_reads` scanner in the ingestion pipeline
What does this MR do and why?
As described in the linked issue, the scanner data can be inconsistent
between the vulnerability_reads
and vulnerability_occurrences
tables for the same vulnerability.
This should never be the case, as they should always be referring to the same data. This is because vulnerability_reads
is just a de-normalized read-optimized table containing heavily read data from the vulnerability
and vulnerability_finding
models
The scanner reference is being properly updated for the finding
in the
finding_map class.
This change makes it so that the existing vulnerability_reads
is also updated with a new scanner reference
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Before | After |
---|---|
before | after |
How to set up and validate locally
prerequisites
- a EE enabled local gdk
- ci runners running locally
- clone the example repo
without the MR patch
-
set the
reports/sast.json
file in the example project to this:example test vulnerability
{ "schema": "https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/raw/v14.0.0/dist/sast-report-format.json", "version": "15.0.0", "scan": { "start_time": "2020-08-24T07:30:25", "end_time": "2020-08-24T07:54:47", "status": "success", "type": "sast", "scanner": { "id": "minac", "name": "Minac manual scanner", "url": "https://gitlab.com/minac", "version": "0.0.1", "vendor": { "name": "Minac" } }, "analyzer": { "id": "minac", "name": "Minac manual analyzer", "version": "0.0.1", "vendor": { "name": "Minac" } } }, "vulnerabilities": [ { "message": "Test vulnerability - 0", "category": "sast", "description": "Test vulnerability description", "id": "d1b90d3d-607b-4b92-a68a-552f7ba83deb", "cve": "d1b90d3d-607b-4b92-a68a-552f7ba83deb", "severity": "Critical", "identifiers": [ { "name": "Test CVE identifier", "type": "CVE", "value": "CVE-2023-XXXX" }, { "name": "CWE identifier", "type": "CWE", "value": "CWE-79" } ], "location": { "file": "README.md", "start_line": 0 }, "scanner": { "id": "minac", "name": "Minac" }, "solution": "No solution" } ], "remediations": [] }
-
run the pipeline for the main branch
- this will seed a vulnerability that has a scanner
external_id
of"minac"
- this will seed a vulnerability that has a scanner
-
run a pipeline again
-
find the vulnerability in the rails console
-
compare
vulnerability.finding.scanner.external_id
andvulnerability.vulnerability_read.scanner.external_id
# find the vulnerability we seeded vulnerability = Vulnerability.find(541) # compare the scanner references. Without the patch, they will be # different. With the patch, they will match vulnerability.finding.scanner.external_id # => "minac-new-version" vulnerability.vulnerability_read.scanner.external_id # => "minac"
with the MR patch
- follow the same steps as above, though this time you should see the
vulnerability.vulnerability_read.scanner
andvulnerability.finding.scanner
stay in sync whenever you edit the scanner id inreports/sast.json
Related to #442929 (closed)
Changelog: fixed
EE: true