Add `purl_type` to SBoM components
Why are we doing this work
Currently, we determine the uniqueness of an SBoM component using its name and the CycloneDX type. As explained in this comment, this is problematic because we may have collisions between packages which have the same name but exist in different package indices (ex: PyPI and Rubygems).
The
component_type
is the type as CycloneDX defines it, meaning that all programming language libraries will have thelibrary
type. PyPI has a package namedpg
and Rubygems also has a package namedpg
. These will both end up having{ component_type: :library, component_name: "pg" }
and will collide, even though they are different dependencies.
As a solution to this, we should add a column onto sbom_components
to store the
package URL type.
We will then change the uniqueness constraint for sbom_components
to be unique by component_type
+ purl_type
+ name
.
Relevant links
Non-functional requirements
-
Documentation: -
Feature flag: -
Performance: -
Testing:
Implementation plan
- Add a
purl_type
column ontosbom_components
. This should be an enum (smallint
) and it should have enums for each of the package url types which are currently supported by dependency scanning. - Remove
index_sbom_components_on_component_type_and_name
- Add a new index on
[:component_type, :purl_type, :name]
- Add a package-url parser to GitLab. Perhaps https://github.com/package-url/packageurl-ruby.
- Update
lib/gitlab/ci/parsers/sbom/cyclonedx.rb
to parse thepurl
from the component and add thepurl_type
to the component object. - Update the
IngestComponents
task to store thepurl_type
and be unique by[:component_type, :purl_type, :name]
Verification steps
-
Enable the feature flag on your project (please comment / tick this in the feature flag rollout issue)
-
Create a new project from a template, use the NodeJS/Express template. Make sure that the group which you create this project under has an ultimate plan.
-
Create a
.gitlab-ci.yml
file with this configuration:include: - template: Security/Dependency-Scanning.gitlab-ci.yml
-
Verify that the
gemnasium-dependency_scanning
outputs agl-sbom-npm-npm.cdx.json
artifact -
Use teleport to connect to the db console. For production, this will require an access request, but this will be automatically sent to the
#infrastructure-lounge
channel when you runtsh login
and is usually approved very quickly. -
Take this query an paste it inside an editor. Replace
YOUR_PIPELINE_ID
with the ID of your CI pipeline which produced thegl-sbom-npm-npm.cdx.json
artifact.select name, version, component_type, purl_type, source_id from sbom_components inner join sbom_component_versions on sbom_components.id = sbom_component_versions.component_id inner join sbom_occurrences on sbom_component_versions.id = sbom_occurrences.component_version_id where pipeline_id = YOUR_PIPELINE_ID;
-
Paste the query with your pipeline ID into the database console. Verify the
purl_type
is present. See https://gitlab.com/gitlab-org/gitlab/-/blob/63a47ad481690a6ecdd9d2f8371ab2c22d2e3fda/app/models/concerns/enums/sbom.rb#L9 for the enum mapping.
Example data
name | version | component_type | purl_type | source_id
----------------------+--------------+----------------+-----------+-----------
@types/babel-types | 7.0.4 | 0 | 6 |
@types/babylon | 6.16.3 | 0 | 6 |
accepts | 1.3.5 | 0 | 6 |
acorn | 3.3.0 | 0 | 6 |
acorn | 4.0.13 | 0 | 6 |
acorn-globals | 3.1.0 | 0 | 6 |
align-text | 0.1.4 | 0 | 6 |
amdefine | 1.0.1 | 0 | 6 |
array-flatten | 1.1.1 | 0 | 6 |
asap | 2.0.6 | 0 | 6 |
asynckit | 0.4.0 | 0 | 6 |
babel-runtime | 6.26.0 | 0 | 6 |
babel-types | 6.26.0 | 0 | 6 |
babylon | 6.18.0 | 0 | 6 |
balanced-match | 1.0.0 | 0 | 6 |
basic-auth | 2.0.0 | 0 | 6 |
body-parser | 1.18.2 | 0 | 6 |
brace-expansion | 1.1.11 | 0 | 6 |
browser-stdout | 1.3.1 | 0 | 6 |
bytes | 3.0.0 | 0 | 6 |
camelcase | 1.2.1 | 0 | 6 |
center-align | 0.1.3 | 0 | 6 |
character-parser | 2.2.0 | 0 | 6 |
clean-css | 3.4.28 | 0 | 6 |
cliui | 2.1.0 | 0 | 6 |
combined-stream | 1.0.6 | 0 | 6 |
commander | 2.15.1 | 0 | 6 |
commander | 2.8.1 | 0 | 6 |
component-emitter | 1.2.1 | 0 | 6 |
concat-map | 0.0.1 | 0 | 6 |
constantinople | 3.1.2 | 0 | 6 |
content-disposition | 0.5.2 | 0 | 6 |
content-type | 1.0.4 | 0 | 6 |
cookie | 0.3.1 | 0 | 6 |
cookie-parser | 1.4.3 | 0 | 6 |
cookie-signature | 1.0.6 | 0 | 6 |
cookiejar | 2.1.2 | 0 | 6 |
core-js | 2.5.7 | 0 | 6 |
core-util-is | 1.0.2 | 0 | 6 |
debug | 2.6.9 | 0 | 6 |
debug | 3.1.0 | 0 | 6 |
decamelize | 1.2.0 | 0 | 6 |
delayed-stream | 1.0.0 | 0 | 6 |
depd | 1.1.1 | 0 | 6 |
depd | 1.1.2 | 0 | 6 |
destroy | 1.0.4 | 0 | 6 |
diff | 3.5.0 | 0 | 6 |
doctypes | 1.1.0 | 0 | 6 |
ee-first | 1.1.1 | 0 | 6 |
encodeurl | 1.0.2 | 0 | 6 |
escape-html | 1.0.3 | 0 | 6 |
escape-string-regexp | 1.0.5 | 0 | 6 |
esutils | 2.0.2 | 0 | 6 |
etag | 1.8.1 | 0 | 6 |
express | 4.16.3 | 0 | 6 |
extend | 3.0.2 | 0 | 6 |
finalhandler | 1.1.1 | 0 | 6 |
form-data | 2.3.2 | 0 | 6 |
formidable | 1.2.1 | 0 | 6 |
forwarded | 0.1.2 | 0 | 6 |
fresh | 0.5.2 | 0 | 6 |
fs.realpath | 1.0.0 | 0 | 6 |
function-bind | 1.1.1 | 0 | 6 |
glob | 7.1.2 | 0 | 6 |
graceful-readlink | 1.0.1 | 0 | 6 |
growl | 1.10.5 | 0 | 6 |
has | 1.0.3 | 0 | 6 |
has-flag | 3.0.0 | 0 | 6 |
he | 1.1.1 | 0 | 6 |
http-errors | 1.6.2 | 0 | 6 |
http-errors | 1.6.3 | 0 | 6 |
iconv-lite | 0.4.19 | 0 | 6 |
inflight | 1.0.6 | 0 | 6 |
inherits | 2.0.3 | 0 | 6 |
ipaddr.js | 1.8.0 | 0 | 6 |
is-buffer | 1.1.6 | 0 | 6 |
is-expression | 3.0.0 | 0 | 6 |
is-promise | 2.1.0 | 0 | 6 |
is-regex | 1.0.4 | 0 | 6 |
isarray | 1.0.0 | 0 | 6 |
js-stringify | 1.0.2 | 0 | 6 |
jstransformer | 1.0.0 | 0 | 6 |
kind-of | 3.2.2 | 0 | 6 |
lazy-cache | 1.0.4 | 0 | 6 |
lodash | 4.17.10 | 0 | 6 |
longest | 1.0.1 | 0 | 6 |
media-typer | 0.3.0 | 0 | 6 |
merge-descriptors | 1.0.1 | 0 | 6 |
methods | 1.1.2 | 0 | 6 |
mime | 1.4.1 | 0 | 6 |
mime-db | 1.35.0 | 0 | 6 |
mime-types | 2.1.19 | 0 | 6 |
minimatch | 3.0.4 | 0 | 6 |
minimist | 0.0.8 | 0 | 6 |
mkdirp | 0.5.1 | 0 | 6 |
mocha | 5.2.0 | 0 | 6 |
morgan | 1.9.0 | 0 | 6 |
ms | 2.0.0 | 0 | 6 |
negotiator | 0.6.1 | 0 | 6 |
object-assign | 4.1.1 | 0 | 6 |
on-finished | 2.3.0 | 0 | 6 |
on-headers | 1.0.1 | 0 | 6 |
once | 1.4.0 | 0 | 6 |
parseurl | 1.3.2 | 0 | 6 |
path-is-absolute | 1.0.1 | 0 | 6 |
path-parse | 1.0.5 | 0 | 6 |
path-to-regexp | 0.1.7 | 0 | 6 |
process-nextick-args | 2.0.0 | 0 | 6 |
promise | 7.3.1 | 0 | 6 |
proxy-addr | 2.0.4 | 0 | 6 |
pug | 2.0.0-beta11 | 0 | 6 |
pug-attrs | 2.0.3 | 0 | 6 |
pug-code-gen | 1.1.1 | 0 | 6 |
pug-error | 1.3.2 | 0 | 6 |
pug-filters | 2.1.5 | 0 | 6 |
pug-lexer | 3.1.0 | 0 | 6 |
pug-linker | 2.0.3 | 0 | 6 |
pug-load | 2.0.11 | 0 | 6 |
pug-parser | 2.0.2 | 0 | 6 |
pug-runtime | 2.0.4 | 0 | 6 |
pug-strip-comments | 1.0.3 | 0 | 6 |
pug-walk | 1.1.7 | 0 | 6 |
qs | 6.5.1 | 0 | 6 |
range-parser | 1.2.0 | 0 | 6 |
raw-body | 2.3.2 | 0 | 6 |
readable-stream | 2.3.6 | 0 | 6 |
regenerator-runtime | 0.11.1 | 0 | 6 |
repeat-string | 1.6.1 | 0 | 6 |
resolve | 1.8.1 | 0 | 6 |
right-align | 0.1.3 | 0 | 6 |
safe-buffer | 5.1.1 | 0 | 6 |
send | 0.16.2 | 0 | 6 |
serve-static | 1.13.2 | 0 | 6 |
setprototypeof | 1.0.3 | 0 | 6 |
setprototypeof | 1.1.0 | 0 | 6 |
source-map | 0.4.4 | 0 | 6 |
source-map | 0.5.7 | 0 | 6 |
statuses | 1.4.0 | 0 | 6 |
string_decoder | 1.1.1 | 0 | 6 |
superagent | 3.8.2 | 0 | 6 |
supertest | 3.1.0 | 0 | 6 |
supports-color | 5.4.0 | 0 | 6 |
to-fast-properties | 1.0.3 | 0 | 6 |
token-stream | 0.0.1 | 0 | 6 |
type-is | 1.6.16 | 0 | 6 |
uglify-js | 2.8.29 | 0 | 6 |
uglify-to-browserify | 1.0.2 | 0 | 6 |
unpipe | 1.0.0 | 0 | 6 |
util-deprecate | 1.0.2 | 0 | 6 |
utils-merge | 1.0.1 | 0 | 6 |
vary | 1.1.2 | 0 | 6 |
void-elements | 2.0.1 | 0 | 6 |
window-size | 0.1.0 | 0 | 6 |
with | 5.1.1 | 0 | 6 |
wordwrap | 0.0.2 | 0 | 6 |
wrappy | 1.0.2 | 0 | 6 |
yargs | 3.10.0 | 0 | 6 |
(157 rows)