Switch to NVD API for advisory generation

Problem to solve

At the moment, we download the NVD JSON feed on a daily basis with https://gitlab.com/gitlab-org/secure/vulnerability-research/advisories/nvd-mirror-json which we then use for advisory generation. This limitation can delay advisories in two ways:

  1. time frame NVD requires to update the file on their end (every 12h)
  2. time frame in which the files are picked up by our scheduled pipeline (every 12h)

By switching over the the NVD API and by increasing the frequency with which we are checking for updates, we should be able to eliminate delay 1 and to reduce delay 2 significantly so ideally we could reduce the delay by 24h.

Proposal

Use the NVD API to include the most recent CVEs into our advisory workflow. For example when requesting all advisories that have been published on Dec, 8th, the API request https://services.nvd.nist.gov/rest/json/cves/2.0/?pubStartDate=2022-12-08T00:00:00.000-05:00&pubEndDate=2022-12-08T23:59:59.999-05:00 gives us access to CVEs that were not available in the JSON feed that was publicly available at the same time.

We can measure the improvement by checking the time delay, i.e., difference between the publication date and merge date. We should be able to observe a drop in the delay after switching over to the API.

Intended users

Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/

/cc @gitlab-org/secure/vulnerability-research @mark.art @sam.white @wayne

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by 🤖 GitLab Bot 🤖