Spike: estimate resource usage for advisory ingestion

Time-box: 3 days.

Topic to Evaluate

In order to implement Ingest Dependency Scanning advisories (&8025 - closed), we need to know what the increase in resource usage will be.

In order to educate our implementation plan, we need to estimate the data storage increase, as well as the number of DB reads/writes.

Tasks to Evaluate

• Advisory databases. Based on Gemnasium and the existing sources utilised by security scanners.
  • Estimate size of initial advisory DB
  • Estimate DB rate of growth based on number of new advisories per year
  • Estimate rate of change for advisory DB (to make it simple, we could say that we update 6x a day - ie every 4 hours)
• Estimate total DB initial size
• Estimate total number of records
• Estimate number of DB reads and writes

References

• Prior work: Spike: estimate resource usage for SBOM ingestion (#361826 - closed)
• Blog post about advisories SemVer
• Advisories SemVer Parser

Risks and Implementation Considerations

• fixed_version field on advisories has a separate syntax for each package manager. We may wish to implement Use unified affected ranges in Gemnasium Vulner... (#220286) in order to make version-matching more straightforward.