## Why are we doing this work This work covers the addition of a service responsible for discovering vulnerable SBOM components, and adding vulnerabilities to the corresponding projects. This happens after a new security advisories is added to the advisory database, or when it's affected range change. **NOTE:** A project might depend on multiple version of a package, so a newly ingested advisory might match multiple SBOM components of the same project. ## Relevant links ## Non-functional requirements * [x] Documentation: https://gitlab.com/gitlab-org/gitlab/-/issues/416075+ * [x] Feature flag: `dependency_scanning_on_advisory_ingestion` * [x] Performance: Check load of background workers when ingesting new advisories. * [x] Testing: Unit tests + verification steps ## Implementation plan Pre-requesite: https://gitlab.com/gitlab-org/gitlab/-/issues/420042+ * [x] ~backend Add `AdvisoryScanner` to [`Gitlab::VulnerabilityScanning`](https://gitlab.com/gitlab-org/gitlab/-/tree/7a31e879c250bc896fb42e76cf8eb45a9827f43e/ee/lib/gitlab/vulnerability_scanning). - For each [`affected_packages`](https://gitlab.com/gitlab-org/gitlab/-/blob/e5348b5afd7dcda1195eb1933fdee9662a4734a8/ee/app/models/package_metadata/advisory.rb#L9) of the advisory, - Fetch batch of affected SBOM components (occurrences) using https://gitlab.com/gitlab-org/gitlab/-/issues/420042+. - Filter out components that don't match the `affected_range` using https://gitlab.com/gitlab-org/gitlab/-/issues/371995. Use memory cache. - Filter out components that belong to a project that doesn't have the feature enabled. - Create vulnerabilities using `Vulnerabilities::ContinuousScan::CreateService`. * [x] ~backend Add `CreateVulnerabilityService`. This creates a single vulnerability and vulnerability for a single affected component detected in a project. - Input: project, pipeline, location, affected component, advisory. - Build a report finding for the advisory and affected component. - If possible, refactor to reuse `Vulnerability::FindingBuilder`. https://gitlab.com/gitlab-org/gitlab/-/issues/371065#note_1494210128 - Convert report finding to custom finding map (w/o security finding). See https://gitlab.com/gitlab-org/gitlab/-/issues/371065#note_1493266758 - Ingest finding map using `Security::Ingestion::IngestReportSliceService`. ## Follow-up issues - **Ingest vulnerabilities from multiple projects at once**, to improve the performance. https://gitlab.com/gitlab-org/gitlab/-/issues/420768 - **Auto-dismiss:** Dismiss an existing vulnerability matching the advisory if the SBOM component version is no longer affected. ## Verification steps We have to temper with the advisories ingested by the backend to test this, so it can't be **tested** on production. However, it can be **checked** on production by adding affected dependencies right before ingesting the corresponding advisory from the GitLab Advisory Database. On a staging environment, - Set up a project supported by Dependency Scanning (DS) and with vulnerable dependencies. - Include DS CI template, and trigger a pipeline for default branch. - Check vulnerabilities created from report or using https://gitlab.com/groups/gitlab-org/-/epics/8026+. - vulnerability report - pipeline page - Export a new advisory that match a project dependency. 2 options: - Add a new vulnerability to the License DB exports (dev). - Prepare a NDJSON export file, and use instructions to sync offline instances. - Wait for the ingestion of the new advisory. - Check vulnerabilities created during the ingestion. - vulnerability report - pipeline page - MR widget if it uses security findings; see https://gitlab.com/gitlab-org/gitlab/-/issues/390185+ <!-- start-discoto-summary --> ## Auto-Summary :robot: <details> <summary>Discoto Usage</summary> --- > **Points** > > Discussion points are declared by headings, list items, and single > lines that start with the text (case-insensitive) `point:`. For > example, the following are all valid points: > > * `#### POINT: This is a point` > * `* point: This is a point` > * `+ Point: This is a point` > * `- pOINT: This is a point` > * `point: This is a **point**` > > Note that any markdown used in the point text will also be propagated > into the topic summaries. > > **Topics** > > Topics can be stand-alone and contained within an issuable (epic, > issue, MR), or can be inline. > > Inline topics are defined by creating a new thread (discussion) > where the first line of the first comment is a heading that starts > with (case-insensitive) `topic:`. For example, the following are all > valid topics: > > * `# Topic: Inline discussion topic 1` > * `## TOPIC: **{+A Green, bolded topic+}**` > * `### tOpIc: Another topic` > > **Quick Actions** > > | Action | Description | > |-------------------------------|---------------------------------------------------------| > | `/discuss sub-topic TITLE` | Create an issue for a sub-topic. Does not work in epics | > | `/discuss link ISSUABLE-LINK` | Link an issuable as a child of this discussion | > --- </details> Last updated by [this job](https://gitlab.com/gitlab-org/secure/pocs/discoto-runner/-/jobs/4783786874) * **TOPIC** Reuse of the FindingBuilder :warning: https://gitlab.com/gitlab-org/gitlab/-/issues/371065#note_1483236612 * **TOPIC** Upsert batches of vulnerability findings :warning: https://gitlab.com/gitlab-org/gitlab/-/issues/371065#note_1491418051 * **TOPIC** Upsert vulnerabilities :white_check_mark: https://gitlab.com/gitlab-org/gitlab/-/issues/371065#note_1492243381 * **TOPIC** IngestReportSliceService and custom finding map :white_check_mark: https://gitlab.com/gitlab-org/gitlab/-/issues/371065#note_1493266758 * **TOPIC** Potential deadlocks when ingesting vulnerabilities https://gitlab.com/gitlab-org/gitlab/-/issues/371065#note_1495943793 <!-- end-discoto-summary --> <!-- start-discoto-topic-settings --><details> <summary>Discoto Settings</summary> <br/> ```yaml --- summary: max_items: -1 sort_by: created sort_direction: ascending ``` See the [settings schema](https://gitlab.com/gitlab-org/secure/pocs/discussion-automation#settings-schema) for details. </details> <!-- end-discoto-topic-settings -->