Change data format of exported advisories
Why are we doing this work
advisory-exporter
currently exports advisory data according to the Gemnasium DB schema. However the Rails backend expects data in a different format. In order to have a successful E2E advisory ingestion we should export data in the right format.
Relevant links
- Object format: #406323 (closed)
- #406836 (closed)
- Identifier object go code: https://gitlab.com/gitlab-org/security-products/analyzers/report/-/blob/main/identifier.go#L74
- Identifier Object Schema: https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/validators/json_schemas/pm_advisory_identifiers.json
Non-functional requirements
-
Documentation: -
Performance: -
Testing:
Data format
Below you can see the current format of a CVE:
Current situation
{
"date": "2023-06-07",
"urls": [
"https://nvd.nist.gov/vuln/detail/CVE-2023-32067",
"https://github.com/c-ares/c-ares/releases/tag/cares-1_19_1",
"https://github.com/c-ares/c-ares/security/advisories/GHSA-9g78-jv2r-p7vc",
"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/B5Z5XFNXTNPTCBBVXFDNZQVLLIE6VRBY/",
"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/UBFWILTA33LOSV23P44FGTQQIDRJHIY7/"
],
"uuid": "559c87d6-cabd-4ed9-ad29-0cbcd31a7954",
"title": "Uncontrolled Resource Consumption",
"cvss_v3": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H",
"cwe_ids": [
"CWE-1035",
"CWE-400",
"CWE-937"
],
"pubdate": "2023-05-25",
"solution": "Upgrade to version 1.19.1 or above.",
"identifier": "CVE-2023-32067",
"description": "c-ares is an asynchronous resolver library. c-ares is vulnerable to denial of service. If a target resolver sends a query, the attacker forges a malformed UDP packet with a length of 0 and returns them to the target resolver. The target resolver erroneously interprets the 0 length as a graceful shutdown of the connection. This issue has been patched in version 1.19.1.",
"identifiers": [
"CVE-2023-32067",
"GHSA-9g78-jv2r-p7vc"
],
"not_impacted": "All versions starting from 1.19.1",
"package_slug": "conan/c-ares",
"affected_range": "<1.19.1",
"fixed_versions": [
"1.19.1"
],
"affected_versions": "All versions before 1.19.1"
}
Desired situation
{
"advisory": {
"id": "559c87d6-cabd-4ed9-ad29-0cbcd31a7954",
"source": "glad",
"title": "Uncontrolled Resource Consumption",
"description": "c-ares is an asynchronous resolver library. c-ares is vulnerable to denial of service. If a target resolver sends a query, the attacker forges a malformed UDP packet with a length of 0 and returns them to the target resolver. The target resolver erroneously interprets the 0 length as a graceful shutdown of the connection. This issue has been patched in version 1.19.1.",
"cvss_v3": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H",
"cvss_v2": "",
"published_date": "2023-05-25",
"urls": [
"https://nvd.nist.gov/vuln/detail/CVE-2023-32067",
"https://github.com/c-ares/c-ares/releases/tag/cares-1_19_1",
"https://github.com/c-ares/c-ares/security/advisories/GHSA-9g78-jv2r-p7vc",
"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/B5Z5XFNXTNPTCBBVXFDNZQVLLIE6VRBY/",
"https://lists.fedoraproject.org/archives/list/package-announce@lists.fedoraproject.org/message/UBFWILTA33LOSV23P44FGTQQIDRJHIY7/"
],
"identifiers": [
{
"type": "cve",
"name": "CVE-2018-1000620",
"value": "CVE-2018-1000620",
"url": "https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-1000620"
},
{
"type": "ghsa",
"name": "GHSA-9g78-jv2r-p7vc",
"value": "GHSA-9g78-jv2r-p7vc",
"url": "https://github.com/advisories/GHSA-9g78-jv2r-p7vc"
},
{
"type": "cwe",
"name": "CWE-1035",
"value": "CWE-1035",
"url": "https://cwe.mitre.org/data/definitions/CWE-1035.html"
},
{
"type": "cwe",
"name": "CWE-400",
"value": "CWE-400",
"url": "https://cwe.mitre.org/data/definitions/CWE-400.html"
},
{
"type": "cwe",
"name": "CWE-937",
"value": "CWE-937",
"url": "https://cwe.mitre.org/data/definitions/CWE-937.html"
}
]
},
"packages": [
{
"name": "c-ares",
"purl_type": "conan",
"affected_range": "<1.19.1",
"solution": "Upgrade to version 1.19.1 or above.",
"fixed_versions": [
"1.19.1"
]
}
]
}
The new advisory schema is composed of two fields:
-
advisory
: Contains the main information about the advisory. More specifically it contains the following fields:-
id
: refers to theuuid
of the advisory. -
source
: this will be set toglad
representing the Gitlab Advisory DB -
title
: thetitle
of the advisory -
description
: thedescription
field of the advisory -
cvss_v2
,cvss_v3
: these are optional fields. If empty they can be ommited -
published_date
: this field maps to thepubdate
field of the advisory. -
urls
: maps to theurl
field in the advisory -
identifiers
: this is an array containing theidentifiers
array and thecwe_ids
array from the advisory. Both identifiers and cwe_ids are array of strings in the advisory. These string values need to become identifier objects according to this schema. We can reuse this code to generate these objects.
-
-
packages
: this field contains information about the advisory packages. For this issue we have only gitlab advisory db as a source. That being saidpackages
is an array but it will contain only one package. More specifically it contains the following fields:-
affected_range
: maps to theaffected_range
field from the advisory -
solution
: maps to thesolution
field from the advisory -
fixed_versions
: maps to thefixed_versions
field from the advisory -
name
: the name of the package. This can be derived from thepackage_slug
field of the advisory. Thepackage_slug
is in the form ofpurl_type/package_name
. -
purl_type
: thepurl_type
of the package. This can be derived from thepackage_slug
field of the advisory which is in the form ofpurl_type/package_name
. We need to be a bit careful with the purl_type since the Gitlab Advisory DB uses different types for some package managers from what the Rails Backend is expecting. We should make sure that we use the value in the first column from the following table. For example if we have a go advisory we should have apurl_type
ofgo
.
-
PURL | gemnasium-db |
---|---|
conan | conan |
gem | gem |
golang | go |
maven | maven |
npm | npm |
nuget | nuget |
composer | packagist |
pypi | pypi |
Implementation Plan
-
Create a function that receives an advisory as input and provides the expected format as the output -
Unit tests for this function -
Document the new structure of the data in the Readme file -
Add a contributing file according to this comment
Edited by Nick Ilieskou