Update cyclonedx 1.5 override
Problem to solve
In Container Scanning CycloneDX reports are not in... (#431406 - closed) • Igor Frenkel • 16.7 we added an override for the cyclonedx
1.5 sboms generated by trivy
in order to ensure that those sbom components are ingested and processed.
However, this is a brittle fix and needs to be removed to either be made more robust or to support cyclonedx 1.5.
Proposal
Make sbom
report conversion robust and permanent until support for specVersion
1.5
is added in 17.0
.
Conversions necessary based on the differences between 1.4 and 1.5
The most stripped down set of attributes was chosen based on what is used in the monolith:
-
metadata
- tools <- has differences between 1.4 and 1.5
- authors
- properties
-
components <- new types in 1.5 which need to be filtered
- bom-ref
- type
- name
- purl
- version
- properties
-
dependencies
- ref
- dependsOn
The differences between 1.4 and 1.5
We only extract needed data that is present in 1.4. This is the list of conversions necessary:
metadata.tools - new object defining creation tools
In 1.5
it is either a new object titled "Creation Tools" or an array of type tool
which is present in 1.4
(only use the latter).
Difference
diff -u <(jq '.definitions.metadata.properties.tools' bom-1.4.schema.json) <(jq'.definitions.metadata.properties.tools' bom-1.5.schema.json)
--- /dev/fd/11 2024-03-19 12:07:48
+++ /dev/fd/13 2024-03-19 12:07:48
@@ -1,9 +1,38 @@
{
- "type": "array",
- "title": "Creation Tools",
- "description": "The tool(s) used in the creation of the BOM.",
- "additionalItems": false,
- "items": {
- "$ref": "#/definitions/tool"
- }
+ "oneOf": [
+ {
+ "type": "object",
+ "title": "Creation Tools",
+ "description": "The tool(s) used in the creation of the BOM.",
+ "additionalProperties": false,
+ "properties": {
+ "components": {
+ "type": "array",
+ "items": {
+ "$ref": "#/definitions/component"
+ },
+ "uniqueItems": true,
+ "title": "Components",
+ "description": "A list of software and hardware components used as tools"
+ },
+ "services": {
+ "type": "array",
+ "items": {
+ "$ref": "#/definitions/service"
+ },
+ "uniqueItems": true,
+ "title": "Services",
+ "description": "A list of services used as tools. This may include microservices, function-as-a-service, and other types of network or intra-process services."
+ }
+ }
+ },
+ {
+ "type": "array",
+ "title": "Creation Tools (legacy)",
+ "description": "[Deprecated] The tool(s) used in the creation of the BOM.",
+ "items": {
+ "$ref": "#/definitions/tool"
+ }
+ }
+ ]
}
metadata.authors - new bom-ref attribute
Difference in child ref organizationalContact
. There is also a difference in that additionalItems: false
was removed in 1.5
but it was incorrectly used so has no effect on conversion (there was only one schema defined for the item)
Difference
diff -u <(jq '.definitions.metadata.properties.authors' bom-1.4.schema.json) <(jq '.definitions.metadata.properties.authors' bom-1.5.schema.json)
--- /dev/fd/11 2024-03-19 12:16:41
+++ /dev/fd/13 2024-03-19 12:16:41
@@ -2,7 +2,6 @@
"type": "array",
"title": "Authors",
"description": "The person(s) who created the BOM. Authors are common in BOMs created through manual processes. BOMs created through automated means may not have authors.",
- "additionalItems": false,
"items": {
"$ref": "#/definitions/organizationalContact"
}
Definition
jq '.definitions.metadata.properties.authors' bom-1.4.schema.json
{
"type": "array",
"title": "Authors",
"description": "The person(s) who created the BOM. Authors are common in BOMs created through manual processes. BOMs created through automated means may not have authors.",
"additionalItems": false,
"items": {
"$ref": "#/definitions/organizationalContact"
}
}
organizationContact
definition in 1.4
jq '.definitions.organizationalContact' bom-1.4.schema.json
{
"type": "object",
"title": "Organizational Contact Object",
"description": "",
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"title": "Name",
"description": "The name of a contact",
"examples": [
"Contact name"
]
},
"email": {
"type": "string",
"format": "idn-email",
"title": "Email Address",
"description": "The email address of the contact.",
"examples": [
"firstname.lastname@example.com"
]
},
"phone": {
"type": "string",
"title": "Phone",
"description": "The phone number of the contact.",
"examples": [
"800-555-1212"
]
}
}
}
For organizationalContact
bom-ref
is added in 1.5
so it has to be excluded in the conversion.
--- /dev/fd/11 2024-03-19 12:19:24
+++ /dev/fd/13 2024-03-19 12:19:24
@@ -4,6 +4,11 @@
"description": "",
"additionalProperties": false,
"properties": {
+ "bom-ref": {
+ "$ref": "#/definitions/refType",
+ "title": "BOM Reference",
+ "description": "An optional identifier which can be used to reference the object elsewhere in the BOM. Every bom-ref MUST be unique within the BOM."
+ },
"name": {
"type": "string",
"title": "Name",
metadata.properties - no change
There is a difference in that additionalItems: false
was removed in 1.5
but it was incorrectly used so has no effect on conversion.
Difference
diff -u <(jq '.definitions.metadata.properties.properties' bom-1.4.schema.json) <(jq '.definitions.metadata.properties.properties' bom-1.5.schema.json)
--- /dev/fd/11 2024-03-19 12:25:55
+++ /dev/fd/13 2024-03-19 12:25:55
@@ -2,7 +2,6 @@
"type": "array",
"title": "Properties",
"description": "Provides the ability to document properties in a name-value store. This provides flexibility to include data not officially supported in the standard without having to use additional namespaces or create extensions. Unlike key-value stores, properties support duplicate names, each potentially having different values. Property names of interest to the general public are encouraged to be registered in the [CycloneDX Property Taxonomy](https://github.com/CycloneDX/cyclonedx-property-taxonomy). Formal registration is OPTIONAL.",
- "additionalItems": false,
"items": {
"$ref": "#/definitions/property"
}
No child schema changes (only description text)
jq '.definitions.property' bom-1.4.schema.json
{
"type": "object",
"title": "Lightweight name-value pair",
"properties": {
"name": {
"type": "string",
"title": "Name",
"description": "The name of the property. Duplicate names are allowed, each potentially having a different value."
},
"value": {
"type": "string",
"title": "Value",
"description": "The value of the property."
}
}
}
diff -u <(jq '.definitions.property' bom-1.4.schema.json) <(jq '.definitions.property' bom-1.5.schema.json)
--- /dev/fd/11 2024-03-19 12:28:10
+++ /dev/fd/13 2024-03-19 12:28:10
@@ -1,6 +1,7 @@
{
"type": "object",
"title": "Lightweight name-value pair",
+ "description": "Provides the ability to document properties in a name-value store. This provides flexibility to include data not officially supported in the standard without having to use additional namespaces or create extensions. Unlike key-value stores, properties support duplicate names, each potentially having different values. Property names of interest to the general public are encouraged to be registered in the [CycloneDX Property Taxonomy](https://github.com/CycloneDX/cyclonedx-property-taxonomy). Formal registration is OPTIONAL.",
"properties": {
"name": {
"type": "string",
components: new component types
Only difference is new component type
being added. We only need library
, container
, and operating-system
. For properties
additionalItems: false
was removed in 1.5
but it was incorrectly used so has no effect on conversion.
#!/bin/bash
paths=$(cat <<- EOS
.definitions.component.properties.["bom-ref"]
.definitions.component.properties.type
.definitions.component.properties.name
.definitions.component.properties.purl
.definitions.component.properties.version
.definitions.component.properties.properties
EOS
)
for path in $paths; do
echo $path
diff -u <(jq "$path" bom-1.4.schema.json) <(jq "$path" bom-1.5.schema.json)
echo
done
.definitions.component.properties.["bom-ref"]
.definitions.component.properties.type
--- /dev/fd/63 2024-03-19 12:42:07
+++ /dev/fd/62 2024-03-19 12:42:07
@@ -5,13 +5,17 @@
"framework",
"library",
"container",
+ "platform",
"operating-system",
"device",
+ "device-driver",
"firmware",
- "file"
+ "file",
+ "machine-learning-model",
+ "data"
],
"title": "Component Type",
- "description": "Specifies the type of component. For software components, classify as application if no more specific appropriate classification is available or cannot be determined for the component. Types include:\n\n* __application__ = A software application. Refer to [https://en.wikipedia.org/wiki/Application_software](https://en.wikipedia.org/wiki/Application_software) for information about applications.\n* __framework__ = A software framework. Refer to [https://en.wikipedia.org/wiki/Software_framework](https://en.wikipedia.org/wiki/Software_framework) for information on how frameworks vary slightly from libraries.\n* __library__ = A software library. Refer to [https://en.wikipedia.org/wiki/Library_(computing)](https://en.wikipedia.org/wiki/Library_(computing))\n for information about libraries. All third-party and open source reusable components will likely be a library. If the library also has key features of a framework, then it should be classified as a framework. If not, or is unknown, then specifying library is RECOMMENDED.\n* __container__ = A packaging and/or runtime format, not specific to any particular technology, which isolates software inside the container from software outside of a container through virtualization technology. Refer to [https://en.wikipedia.org/wiki/OS-level_virtualization](https://en.wikipedia.org/wiki/OS-level_virtualization)\n* __operating-system__ = A software operating system without regard to deployment model (i.e. installed on physical hardware, virtual machine, image, etc) Refer to [https://en.wikipedia.org/wiki/Operating_system](https://en.wikipedia.org/wiki/Operating_system)\n* __device__ = A hardware device such as a processor, or chip-set. A hardware device containing firmware SHOULD include a component for the physical hardware itself, and another component of type 'firmware' or 'operating-system' (whichever is relevant), describing information about the software running on the device.\n* __firmware__ = A special type of software that provides low-level control over a devices hardware. Refer to [https://en.wikipedia.org/wiki/Firmware](https://en.wikipedia.org/wiki/Firmware)\n* __file__ = A computer file. Refer to [https://en.wikipedia.org/wiki/Computer_file](https://en.wikipedia.org/wiki/Computer_file) for information about files.",
+ "description": "Specifies the type of component. For software components, classify as application if no more specific appropriate classification is available or cannot be determined for the component. Types include:\n\n* __application__ = A software application. Refer to [https://en.wikipedia.org/wiki/Application_software](https://en.wikipedia.org/wiki/Application_software) for information about applications.\n* __framework__ = A software framework. Refer to [https://en.wikipedia.org/wiki/Software_framework](https://en.wikipedia.org/wiki/Software_framework) for information on how frameworks vary slightly from libraries.\n* __library__ = A software library. Refer to [https://en.wikipedia.org/wiki/Library_(computing)](https://en.wikipedia.org/wiki/Library_(computing))\n for information about libraries. All third-party and open source reusable components will likely be a library. If the library also has key features of a framework, then it should be classified as a framework. If not, or is unknown, then specifying library is RECOMMENDED.\n* __container__ = A packaging and/or runtime format, not specific to any particular technology, which isolates software inside the container from software outside of a container through virtualization technology. Refer to [https://en.wikipedia.org/wiki/OS-level_virtualization](https://en.wikipedia.org/wiki/OS-level_virtualization)\n* __platform__ = A runtime environment which interprets or executes software. This may include runtimes such as those that execute bytecode or low-code/no-code application platforms.\n* __operating-system__ = A software operating system without regard to deployment model (i.e. installed on physical hardware, virtual machine, image, etc) Refer to [https://en.wikipedia.org/wiki/Operating_system](https://en.wikipedia.org/wiki/Operating_system)\n* __device__ = A hardware device such as a processor, or chip-set. A hardware device containing firmware SHOULD include a component for the physical hardware itself, and another component of type 'firmware' or 'operating-system' (whichever is relevant), describing information about the software running on the device.\n See also the list of [known device properties](https://github.com/CycloneDX/cyclonedx-property-taxonomy/blob/main/cdx/device.md).\n* __device-driver__ = A special type of software that operates or controls a particular type of device. Refer to [https://en.wikipedia.org/wiki/Device_driver](https://en.wikipedia.org/wiki/Device_driver)\n* __firmware__ = A special type of software that provides low-level control over a devices hardware. Refer to [https://en.wikipedia.org/wiki/Firmware](https://en.wikipedia.org/wiki/Firmware)\n* __file__ = A computer file. Refer to [https://en.wikipedia.org/wiki/Computer_file](https://en.wikipedia.org/wiki/Computer_file) for information about files.\n* __machine-learning-model__ = A model based on training data that can make predictions or decisions without being explicitly programmed to do so.\n* __data__ = A collection of discrete values that convey information.",
"examples": [
"library"
]
.definitions.component.properties.name
.definitions.component.properties.purl
.definitions.component.properties.version
.definitions.component.properties.properties
--- /dev/fd/63 2024-03-19 12:42:07
+++ /dev/fd/62 2024-03-19 12:42:07
@@ -2,7 +2,6 @@
"type": "array",
"title": "Properties",
"description": "Provides the ability to document properties in a name-value store. This provides flexibility to include data not officially supported in the standard without having to use additional namespaces or create extensions. Unlike key-value stores, properties support duplicate names, each potentially having different values. Property names of interest to the general public are encouraged to be registered in the [CycloneDX Property Taxonomy](https://github.com/CycloneDX/cyclonedx-property-taxonomy). Formal registration is OPTIONAL.",
- "additionalItems": false,
"items": {
"$ref": "#/definitions/property"
}
dependencies: no change
properties.dependencies
diff -u <(jq '.properties.dependencies' bom-1.4.schema.json) <(jq '.properties.dependencies' bom-1.5.schema.json)
--- /dev/fd/11 2024-03-19 12:47:20
+++ /dev/fd/13 2024-03-19 12:47:20
@@ -1,6 +1,5 @@
{
"type": "array",
- "additionalItems": false,
"items": {
"$ref": "#/definitions/dependency"
},
definitions.dependency
diff -u <(jq '.definitions.dependency' bom-1.4.schema.json) <(jq '.definitions.dependency' bom-1.5.schema.json)
--- /dev/fd/11 2024-03-19 12:46:38
+++ /dev/fd/13 2024-03-19 12:46:38
@@ -1,26 +1,25 @@
{
"type": "object",
"title": "Dependency",
- "description": "Defines the direct dependencies of a component. Components that do not have their own dependencies MUST be declared as empty elements within the graph. Components that are not represented in the dependency graph MAY have unknown dependencies. It is RECOMMENDED that implementations assume this to be opaque and not an indicator of a component being dependency-free.",
+ "description": "Defines the direct dependencies of a component or service. Components or services that do not have their own dependencies MUST be declared as empty elements within the graph. Components or services that are not represented in the dependency graph MAY have unknown dependencies. It is RECOMMENDED that implementations assume this to be opaque and not an indicator of a object being dependency-free. It is RECOMMENDED to leverage compositions to indicate unknown dependency graphs.",
"required": [
"ref"
],
"additionalProperties": false,
"properties": {
"ref": {
- "$ref": "#/definitions/refType",
+ "$ref": "#/definitions/refLinkType",
"title": "Reference",
- "description": "References a component by the components bom-ref attribute"
+ "description": "References a component or service by its bom-ref attribute"
},
"dependsOn": {
"type": "array",
"uniqueItems": true,
- "additionalItems": false,
"items": {
- "$ref": "#/definitions/refType"
+ "$ref": "#/definitions/refLinkType"
},
"title": "Depends On",
- "description": "The bom-ref identifiers of the components that are dependencies of this dependency object."
+ "description": "The bom-ref identifiers of the components or services that are dependencies of this dependency object."
}
}
}
1.5
adds a new type refLinkType
which is defined as:
jq '.definitions.refLinkType' bom-1.5.schema.json
{
"description": "Descriptor for an element identified by the attribute 'bom-ref' in the same BOM document.\nIn contrast to `bomLinkElementType`.",
"allOf": [
{
"$ref": "#/definitions/refType"
}
]
}
The difference between refType
in 1.4
and 1.5
is in minLength
and is more stringent in the latter, so it can be used as is.
diff -u <(jq '.definitions.refType' bom-1.4.schema.json) <(jq '.definitions.refType' bom-1.5.schema.json)
--- /dev/fd/11 2024-03-19 12:50:09
+++ /dev/fd/13 2024-03-19 12:50:09
@@ -1,4 +1,6 @@
{
- "$comment": "Identifier-DataType for interlinked elements.",
- "type": "string"
+ "description": "Identifier for referable and therefore interlink-able elements.",
+ "type": "string",
+ "minLength": 1,
+ "$comment": "value SHOULD not start with the BOM-Link intro 'urn:cdx:'"
}
Possible fixes
- Convert the generated
sbom
to a minimum set of required attributes in the analyzer. See proposal: #431435 (comment 1652864215) - Use a 3rd party tool https://github.com/CycloneDX/cyclonedx-cli to convert the
1.5
sbom to1.4
.
Implementation plan
- Update sbom_converter to add
sanitize_components
method that implements a stripped down report from this comment: #431435 (comment 1652864215) - Remove original override code fix_report_version!
Verification
- Ensure that monolith can parse new report
- Add a container scanning job to test project
- Use image built for this issue's MR
- Verify that sbom is imported as expected by the monolith
See MR verification section.