Create command line component for vuln-list ingestion
This will be a command line component that is run from a scheduled pipeline job in the license db deployment project.
The trivy-db is generated from an export of the vuln-list project. This project is a better source for ingestion as it's versioned through commits, allowing new changes to be easily identified (the trivy-db doesn't have any timestamps on its records).
Each Linux distribution has a separate issue for adding support into this command line component.
Data will be ingested into the license db cloud sql instance.
Data Sources to Ignore:
The following data sources will not be ingested due to licensing issues.
- Alpine (license)
- Oracle (license)
Ingestion:
- Clone or pull vuln-list repository
- Identify modified files using the commit history
- Process modified files
For this issue:
- Create a repository that will contain both the
vuln-listingestion component and also the alpine ingestion component. - Create a command line component and proper structure to support ingesting all of data feeds needed.
- Configuration through variables
- Clone or pull the
vuln-listGitHub repository- Make the list of changed files available
- Identify changed files that require processing
- Call various advisory/distro ingestion functions
- Provide interface to update DB
- Store state in DB regarding last ingested commit for
vuln-listrepository
Edited by Michael Eddington