Skip to content

Improve Vulnerability Tracking: Use Signatures [RUN ALL RSPEC] [RUN AS-IF-FOSS]

James Johnson requested to merge improve_vuln_tracking-backend_use_fingerprints into master

This MR is the GitLab second part of the backend implementation of the proof-of-concept MR: !45339 (closed) and is part of the epic https://gitlab.com/groups/gitlab-org/-/epics/4690 - Improve Vulnerability Tracking: MVP: Scope+Offset

Feature flag vulnerability_finding_tracking_signatures rollout issue: #322044 (closed)

What does this MR do?

This merge request uses the vulnerability finding signatures (previously `fingerprints) stored in the database (added in the dependent MR !55173 (merged)) to compare/deduplicate vulnerability signatures.

High-Level Details

This is done by:

  • Using the signatures stored in the vulnerability_finding_signatures table (DB MR: !52720 (merged), Store Signatures MR: !55173 (merged))
  • Comparing signatures of two vulnerability findings in order of decreasing priority
  • Updating existing feedback signature values that match a signature to use the latest, highest priority signature

Technical Details

Below are a few scenarios that are expected to occur when comparing vulnerability signatures using prioritized signatures.

Each tracking signature method has a priority defined in the GitLab backend. Vulnerabilities with matching tracking signatures are compared in descending order.

One Matching Lower-Priority Signature (expand me)

In this example Vuln1's Algorithm:hash tracking signature below is the only match with any of Vuln2's tracking signatures. Since this is the first (and only) match, they are considered the same vulnerability.

graph LR
  subgraph Vuln1
    subgraph signature11[signature1]
      priority11["Priority:2"]
      algorithm_type11["Algorithm:scope_offset"]
      value11["AAAA"]
    end
    subgraph signature12[signature2]
      priority12["Priority:1"]
      algorithm_type12["Algorithm:hash"]
      value12["XXXX"]
    end
  end

  subgraph Vuln2
    subgraph signature22[signature2]
      priority22["Priority:1"]
      algorithm_type22["Algorithm:hash"]
      value22["XXXX"]
    end
  end

  priority12 -->|matches| priority22
  algorithm_type12 -->|matches| algorithm_type22
  value12 -->|matches| value22

  classDef green fill:#9f9,stroke-width:4px,font-weight:bold;
  classDef red fill:#f9f,stroke:#333,stroke-width:4px;

  class signature22 green
  class signature12 green
Two Matching Signatures (expand me)

In this next example, Vuln1 and Vuln2 have several matching signatures. However, the first and highest priority (most accurate) match is with the Algorithm::scope_offset tracking signature:

graph LR
  subgraph Vuln1
    subgraph signature11[signature1]
      priority11["Priority:2"]
      algorithm_type11["Algorithm:scope_offset"]
      value11["AAAA"]
    end
    subgraph signature12[signature2]
      priority12["Priority:1"]
      algorithm_type12["Algorithm:hash"]
      value12["XXXX"]
    end
  end

  subgraph Vuln2
    subgraph signature21[signature1]
      priority21["Priority:2"]
      algorithm_type21["Algorithm:scope_offset"]
      value21["AAAA"]
    end
    subgraph signature22[signature2]
      priority22["Priority:1"]
      algorithm_type22["Algorithm:hash"]
      value22["XXXX"]
    end
  end

  priority11-->|matches| priority21
  algorithm_type11 -->|matches| algorithm_type21
  value11 -->|matches| value21

  priority12-.->|matches| priority22
  algorithm_type12 -.->|matches| algorithm_type22
  value12 -.->|matches| value22

  classDef green fill:#9f9,stroke-width:4px,font-weight:bold;
  classDef red fill:#f9f,stroke:#333,stroke-width:4px;

  class signature11 green
  class signature21 green
Non-Matching High Priority Signatures (expand me)

Prioritized tracking signatures are also used to disprove matches. In the example below, Vuln1 and Vuln2 both have matching low-priority Algorithm:hash signatures, but the high-priority Algorithm:scope_offset tracking signatures do not match

graph LR
  subgraph Vuln1
    subgraph signature11[signature1]
      priority11["Priority:2"]
      algorithm_type11["Algorithm:scope_offset"]
      value11["AAAA"]
    end
    subgraph signature12[signature2]
      priority12["Priority:1"]
      algorithm_type12["Algorithm:hash"]
      value12["XXXX"]
    end
  end

  subgraph Vuln2
    subgraph signature21[signature1]
      priority21["Priority:2"]
      algorithm_type21["Algorithm:scope_offset"]
      value21["BBBB"]
    end
    subgraph signature22[signature2]
      priority22["Priority:1"]
      algorithm_type22["Algorithm:hash"]
      value22["XXXX"]
    end
  end

  priority11-->|"✗"| priority21
  algorithm_type11 -->|"✗"| algorithm_type21
  value11 -->|"✗"| value21

  priority12-.->|matches| priority22
  algorithm_type12 -.->|matches| algorithm_type22
  value12 -.->|matches| value22

  classDef green fill:#9f9,stroke-width:4px,font-weight:bold;
  classDef red fill:#f99,stroke-width:4px,font-weight:bold;

  class signature11 red
  class signature21 red

Notes

The changes in this MR are intended to be gated by a feature flag to make testing/rolling out this new feature easier.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Thiago Figueiró

Merge request reports