Create command line component for vuln-list ingestion

This will be a command line component that is run from a scheduled pipeline job in the license db deployment project.

The trivy-db is generated from an export of the vuln-list project. This project is a better source for ingestion as it's versioned through commits, allowing new changes to be easily identified (the trivy-db doesn't have any timestamps on its records).

Each Linux distribution has a separate issue for adding support into this command line component.

Data will be ingested into the license db cloud sql instance.

Data Sources to Ignore:

The following data sources will not be ingested due to licensing issues.

  1. Alpine (license)
  2. Oracle (license)

Ingestion:

  1. Clone or pull vuln-list repository
  2. Identify modified files using the commit history
  3. Process modified files

For this issue:

  1. Create a repository that will contain both the vuln-list ingestion component and also the alpine ingestion component.
  2. Create a command line component and proper structure to support ingesting all of data feeds needed.
  3. Configuration through variables
  4. Clone or pull the vuln-list GitHub repository
    1. Make the list of changed files available
  5. Identify changed files that require processing
  6. Call various advisory/distro ingestion functions
  7. Provide interface to update DB
  8. Store state in DB regarding last ingested commit for vuln-list repository
Edited by Michael Eddington