Skip to content

feat: Add MarkDroppedAsResolved service to Security Ingestion flow

What does this MR do and why?

Adds ScheduleMarkDroppedAsResolved and MarkDroppedAsResolvedWorker for resolving vulnerabilities tied to identifiers that are no longer detected.

This worker depends on the presence of report.scan.primary_identifiers to get an exhaustive list of all identifiers for which the analyzer scans (as opposed to all identifiers detected).

See related schema update to support scan.primary_identifiers field (not strictly necessary since we use additionalProperties: true) gitlab-org/security-products/security-report-schemas!126 (merged)

Relates to #368284 (closed)

Background

Scenario:

  1. semgrep-sast is updated to remove rule that has been identified as overly noisy and prone to producing high false-positive results
  2. Report includes list of included rules executed during scan
  3. Report parser identifies all existing findings that are not included in list of rules and auto-resolves due to identifier no longer being included
graph LR

repository --> brakeman_sast

subgraph analyzer pipeline
  brakeman_sast --> report1a["noisy-rule-123 dropped"]
  report1a --> report1b["scan.identifiers populated"]
  report1b --> reportout1("gl-sast-report.json")
end

subgraph ingestion
  reportout1 --> ingestsvc["IngestReportService"]

  ingestsvc--> schedulesvc["ScheduleMarkDroppedAsResolved"]
end

subgraph sidekiq
  schedulesvc --> worker["MarkDroppedAsResolvedWorker"]
end

Depends on:

{
  "vulnerabilities": [],
  "scan": {
    "primary_identifiers": [
      {
        "type": "semgrep_id",
        "name": "gosec.G106-1",
        "value": "gosec.G106-1"
      }
    ],
    "status": "success"
  }
}

Screenshots

list details
Screen_Shot_2022-09-13_at_2.45.57_PM Screen_Shot_2022-09-13_at_2.46.10_PM

How to set up and validate locally

Test project export using modified fixtures: 2022-08-16_16-45-709_root_go_export.tar.gz

  1. Enable the feature flag: Feature.enable(:sec_mark_dropped_findings_as_resolved)
  2. Run default pipeline
  3. Confirm presence of 3 vulnerabilities on dashboard
  4. Update .gitlab-ci.yml to reference gl-sast-report.tests-go-with-scan-primary-identifiers.json
  5. Confirm no change in behavior (still 3 vulnerabilities on dashboard)
  6. Update .gitlab-ci.yml to reference gl-sast-report.tests-go-with-scan-primary-identifiers-dropping-G104.json
  7. Confirm auto-resolution of dropped identifier (2 remaining detected vulnerabilities, 1 resolved and no longer detected)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Lucas Charles

Merge request reports