CVS on advisory DB changes (GA)

Why are we doing this work

We need to enable Continuous Vulnerability Scanning (CVS) on advisory DB changes in all Ultimate projects to reach Global Availability (GA).

Related links

Proposal

Candidate: proposal 2. See #426089 (comment 1583304425)

Proposal 1

Introduce a feature flag so that some of the AdvisoryScanWorker jobs apply to all Ultimate projects, instead of being limited to projects where it's been explicitly enabled.

Pros

  • It might be the simplest option.

Cons

  • On a typical day we have very few advisories that trigger scans. As a result, the incremental rollout lack granularity.
  • If a scan job fails (because it checks all projects and runs for too long), then this impacts projects where the experimental feature has been enabled.

Proposal 2

Introduce a feature flag that makes the backend scans two sets of projects in two different jobs:

  • a job that only scans the projects where the feature has been enabled
  • a job that only scans the other Ultimate projects

Implementation

  • Introduce a request-based flag.
  • Keep the existing AdvisoryScanWorker. It's triggered by PackageMetadata::IngestedAdvisoryEvent.
  • Add a similar worker or argument to that worker, so that the same event trigger the scan of all Ultimate projects, except the ones where the feature has been enabled.

Pros

  • Projects where the feature has been enabled aren't impacted by failing jobs.

Cons

  • Similar to Proposal 1, it lacks granularity.
  • It's more complex than Proposal 1.

Proposal 3

Add a worker to process one batch of projects/SBOM occurrences. Make AdvisoryScanWorker responsible for creating batches and calling that worker.

Migration path and rollout are TBD.

Pros

Cons

  • It's more complex than Proposal 1 and Proposal 2.
  • Feasibility is to be checked.
  • It's harder to collect metrics. Issue: #397550 (closed)

Implementation plan

The implementation of proposal 2 is being discussed in #426089 (comment 1583310953).

  1. Introduce global param.
    • Change AdvisoryScanService and AdvisoryScanner to accept a new global parameter.
    • Change AdvisoryScanWorker to set global to false when calling AdvisoryScanService.
    • Change AdvisoryScanner to raise NotImplementedError when global is true.
  2. Start counting matches as if GA.
    • Introduce a new GlobalAdvisoryScanWorker for global scans. It's almost identical to the existing AdvisoryScanWorker but it sets global to true.
    • Update the finder so that it takes a new param named global . When global is true, it no longer applies the filter_by_cvs_enabled scope to the relation.
    • Change AdvisoryScanner to only count possible matches when global is true.
      • Pass global to the finder.
      • Publish event with .counts.possibly_affected_sbom_occurrences.
      • Ensure that the two workers publish to distinct events. TBD.
      • Return early.
  3. Implement global scan.
    • Introduce a new feature flag name global_dependency_scanning_on_advisory_ingestion.
    • Change AdvisoryScanner to do a full scan and create advisories when global is true and global_dependency_scanning_on_advisory_ingestion is enabled.

Auto-Summary 🤖

Discoto Usage

Points

Discussion points are declared by headings, list items, and single lines that start with the text (case-insensitive) point:. For example, the following are all valid points:

  • #### POINT: This is a point
  • * point: This is a point
  • + Point: This is a point
  • - pOINT: This is a point
  • point: This is a **point**

Note that any markdown used in the point text will also be propagated into the topic summaries.

Topics

Topics can be stand-alone and contained within an issuable (epic, issue, MR), or can be inline.

Inline topics are defined by creating a new thread (discussion) where the first line of the first comment is a heading that starts with (case-insensitive) topic:. For example, the following are all valid topics:

  • # Topic: Inline discussion topic 1
  • ## TOPIC: **{+A Green, bolded topic+}**
  • ### tOpIc: Another topic

Quick Actions

Action Description
/discuss sub-topic TITLE Create an issue for a sub-topic. Does not work in epics
/discuss link ISSUABLE-LINK Link an issuable as a child of this discussion

Last updated by this job

Discoto Settings 
---
summary:
  max_items: -1
  sort_by: created
  sort_direction: ascending

See the settings schema for details.

Edited by Fabien Catteau