Recompute inventory stats when vulnerabilities go undetected

What does this MR do and why?

Recompute inventory stats when vulnerabilities go undetected

Security Inventory severity counts are backed by the vulnerability_statistics table. When a pipeline scan stops reporting a previously detected vulnerability, MarkAsResolvedService flips resolved_on_default_branch to true with a bulk update_all. That bulk write bypasses the incremental statistics path, which relies on dirty tracking of a loaded record, so the inventory count stayed stale until the twice-daily AdjustmentWorker recompute corrected it.

Enqueue a per-project AdjustmentWorker run from MarkAsResolvedService when it marks vulnerabilities as no longer detected, so the counts are corrected right away. This reuses the same recompute path the archival, removal, and auto-resolve flows already use for their bulk writes. The enqueue fires once per execute and only when a real transition happened, and it is gated by the security_inventory_no_longer_detected_vulnerabilities flag so the live and batch paths filter consistently.

Re-detection resolved_on_default_branch back to false during ingestion) stays on the cron cadence for now, since those writes run on every pipeline and have no cheap signal for a genuine flip.

Risks and trade-offs

  • Recompute cost: one extra per-project recompute each time a pipeline marks something no longer detected ( once per pipeline ). Bounded by project size and by the no_longer_detected.present? guard. Comparable to the existing archival, removal, and auto-resolve enqueues.
  • Possible duplicate enqueue with auto-resolve in the same run. Harmless; the worker is safe to run twice. AdjustmentWorker is intentionally not idempotent-deduplicated, so we cannot rely on Sidekiq dedup to collapse them; the once-per-execute flag keeps our own contribution to a single enqueue.
  • Re-detection ( resolved_on_default_branch changes from true to false ) stays on the cron cadence.

Rejected alternative

The incremental approach ( calculating and propagating diff instead of the SQL based recompute ) was rejected because the bulk writes that flip the flag bypass the very mechanism it depends on, and replicating that mechanism by hand is strictly more complex and riskier than reusing the existing recompute.

References

#600455

How to set up and validate locally

It is difficult to come up with a plausible testing scenario that doesn't rely to heavy on console operations.

Steps

Console-driven test against a seeded project. It exercises the exact new code (MarkAsResolvedService -> enqueue -> AdjustmentWorker recompute), is deterministic, and avoids CI/scanner flakiness.

Prerequisites

# in rails console
Feature.enable(:security_inventory_no_longer_detected_vulnerabilities)   # global; works for :instance or namespace

Make sure background jobs run so the enqueued recompute executes: gdk start rails-background-jobs.

Step 1 - pick a currently-counted vuln and snapshot the count

project = Project.find_by_full_path('superorg/depb/redteam/megascript')  # any seeded project with criticals

def counts(project)
  project.reload.vulnerability_statistic
    &.slice('total', 'critical', 'high', 'medium', 'low', 'unknown', 'info', 'letter_grade')
end

target = project.vulnerability_reads
  .where(state: Vulnerability.active_states, resolved_on_default_branch: false, severity: :critical)
  .where.not(report_type: :generic)        # generic = manual, the service skips these
  .first

before = counts(project)

Step 2 - run the real service the way ingestion does

scanner         = target.scanner
tracked_context = target.tracked_context
pipeline        = project.ci_pipelines.last

# everything this scanner still reports EXCEPT our target => only target is "missing"
ingested = project.vulnerability_reads.by_scanner(scanner).pluck(:vulnerability_id) - [target.vulnerability_id]

Security::Ingestion::MarkAsResolvedService.execute(pipeline, scanner, ingested, tracked_context)
sleep 3   # let the enqueued AdjustmentWorker run

Step 3 - assert the live correction

[before, counts(project)]

Pass: critical is exactly one lower than before, and you did not run AdjustmentWorker yourself. That is the live path: the service flipped the flag and enqueued the recompute.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Vasyl Pedak

Merge request reports

Loading