No licenses for Python packages in Dependency List when name isn't canonical (SBOM Scanner)
Summary
For some packages, like Django
, a license is found but in Security & Compliance
> Dependency list
the dependency does not show the license. More info can be found here
Further details
The License Scanning SBOM Scanner normalizes names of Python packages found in CycloneDX SBOMs before comparing them to DB column pm_packages.name
. As a result, Django
shows up with a license on the License Compliance page.
However, names are not normalized in the same way when Pipeline#dependency_list_report
merges the DependencyScanning::Report
into the DependencyList::Report
. So Django
ends up being compared to django
; these don't match, and Django
has no licenses on the Dependency List.
Pipeline#dependency_list_report
def apply_license(license)
dependencies.each do |dependency|
next unless license.dependencies.find { |license_dependency| license_dependency.name == dependency[:name] }
next if dependency[:licenses].find { |license_hash| license_hash[:name] == license.name }
dependency[:licenses].push(name: license.name, url: license.url)
end
end
CycloneDX SBOM
{
"bomFormat": "CycloneDX",
"specVersion": "1.4",
"serialNumber": "urn:uuid:9c41660a-ecb7-431e-83c7-7f00c64610b0",
"version": 1,
"metadata": {
"timestamp": "2023-03-01T12:16:45Z",
"tools": [
{
"vendor": "GitLab",
"name": "Gemnasium",
"version": "3.11.3"
}
],
"authors": [
{
"name": "GitLab",
"email": "support@gitlab.com"
}
],
"properties": [
{
"name": "gitlab:dependency_scanning:input_file",
"value": "Pipfile"
},
{
"name": "gitlab:dependency_scanning:package_manager",
"value": "pipenv"
}
]
},
"components": [
{
"name": "Django",
"version": "1.11.3",
"purl": "pkg:pypi/Django@1.11.3",
"type": "library",
"bom-ref": "pkg:pypi/Django@1.11.3"
},
{
"name": "docutils",
"version": "0.13.1",
"purl": "pkg:pypi/docutils@0.13.1",
"type": "library",
"bom-ref": "pkg:pypi/docutils@0.13.1"
},
{
"name": "pytz",
"version": "2022.7.1",
"purl": "pkg:pypi/pytz@2022.7.1",
"type": "library",
"bom-ref": "pkg:pypi/pytz@2022.7.1"
},
{
"name": "requests",
"version": "2.5.3",
"purl": "pkg:pypi/requests@2.5.3",
"type": "library",
"bom-ref": "pkg:pypi/requests@2.5.3"
}
]
}
Steps to reproduce
Clone python-pipenv test project into staging and edit the .gitlab-ci.yml
to:
stages:
- test
include:
- template: Dependency-Scanning.gitlab-ci.yml
Push the updated .gitlab-ci.yml
to your new forked project. After the DS job runs you should be able to see information in:
-
Security & Compliance
>License Compliance
-
Security & Compliance
>Dependency list
Example Project
An example project is python-pipenv-replace-license-finder-test-new-job
What is the current bug behavior?
What is the expected correct behavior?
When a package license is identified and can be viewed under Security & Compliance
> License Compliance
it should also be present under
Security & Compliance
> Dependency list
.
Relevant logs and/or screenshots
License for django 1.11.3 is BSD-3-Clause
But license is missing in dependency list
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
Update DependencyList::Report#apply_license
to normalize package names using Sbom::PackageUrl::Normalizer.
DependencyList::Report#apply_license
is called from DependencyList#apply_license!
.
Implementation plan
-
Update Dependency
to trackpurl_type
-
Update DependencyList::Report#apply_license
to compare normalized package names. -
Update the specs for DependencyList::Report#apply_license
and/or the specs forDependencyList#apply_license!
to cover the edge case.
See ee/spec/lib/gitlab/ci/reports/dependency_list/report_spec.rb and ee/spec/lib/gitlab/ci/parsers/security/dependency_list_spec.rb.