## Background context and Problems to Solve Please reference the [parent epic](https://gitlab.com/groups/gitlab-org/-/epics/7886) for context and details. This epic specifically deals with the ingestion of vulnerabilities from [gemnasium-db](https://gitlab.com/gitlab-org/security-products/gemnasium-db) into the main Postgres database. ## Engineering DRI @ifrenkel (Rails) @nilieskou (PMDB) ## Proposal Ingest vulnerability advisories through License DB. The following tasks are performed on a regular basis: - License DB syncs with gemnasium-db, and updates its own DB. - License DB exports updates to GCP buckets. - The Rails backend syncs with the GCP buckets, and stores data into the `vulnerability_advisories` table. Vulnerability advisories are ingested the same way package licenses are. A new sync protocol has been proposed in https://gitlab.com/gitlab-org/gitlab/-/issues/394723+. The components added to License DB are similar to the ones listed in https://gitlab.com/gitlab-org/security-products/license-db/deployment/-/blob/main/docs/security.md: > * license-feeder: Handles communicating with package registries to get a list of package names. These package names are then sent to the license-interfacer over a secure pub/sub channel. > * license-interfacer: Handles incoming messages to process and execute HTTP(s) requests to package registries. > * license-processor: Handles incoming messages from the interfacer to store the results of license data into a PostgreSQL database. > * license-exporter: Handles exporting data from the database to the Public GCP bucket. ## Implementation plan - License DB - Update the `schema` to store advisories. - Add an `advisory-feeder` that gets new or updated advisories from `gemnasium-db`. - Add a generic `advisory-processor` to store advisories. - Add a generic `advisory-exporter` to export new or updated advisories to GCP buckets. - Rails backend - Update the `vulnerability_advisories` table to accurately reference affected packages. - Add a periodic worker that syncs `vulnerability_advisories` with the export buckets.