Synchronize gemnasium-db with NVD

Problem to solve

As part of maintaining the Gemnasium DB we need to sync with NVD and import security advisories published on NVD after reviewing, and possibly editing them.

Intended users

groupcomposition analysis team members

Further details

See epic's proposal for the expected workflow to implement.

Proposal

• Create a script that processes NVD's RSS feed and/or JSON "feeds"
  • Create a JSON->YAML converter
  • Only process new security advisories (MR)
  • Convert affected range
  • Translate CPE (product and vendor) to package type and name
    • Ignore CPE if in ignore-list
    • Ignore CPE based on list of patterns
    • Resolve CPE automatically using a map
    • Resolve CPE manually it matches a package name
• Integrate script to gemnasium-db
  • Add to scheduled pipelines (stretch)
  • Create MRs automatically
• Bootstrap the CPE to package map using various sources
  • gemnasium-db
  • rubysec
  • FriendsOfPHP
  • CPE Dictionaly
• Expand the README.md with a detailed documentation about the tool and the implemented workflows

Documentation

Update the Sources documentation to mention the automated process.

What does success look like, and how can we measure that?

Advisories from NVD are synchronized with gemnasium-db will still being reviewed by GitLab.

Links / references