No licenses for Python packages in Dependency List when name isn't canonical (SBOM Scanner)

Summary

For some packages, like Django, a license is found but in Security & Compliance > Dependency list the dependency does not show the license. More info can be found here

Further details

The License Scanning SBOM Scanner normalizes names of Python packages found in CycloneDX SBOMs before comparing them to DB column pm_packages.name. As a result, Django shows up with a license on the License Compliance page. ✅

However, names are not normalized in the same way when Pipeline#dependency_list_report merges the DependencyScanning::Report into the DependencyList::Report. So Django ends up being compared to django; these don't match, and Django has no licenses on the Dependency List. ❌

Pipeline#dependency_list_report

          def apply_license(license)
            dependencies.each do |dependency|
              next unless license.dependencies.find { |license_dependency| license_dependency.name == dependency[:name] }
              next if dependency[:licenses].find { |license_hash| license_hash[:name] == license.name }

              dependency[:licenses].push(name: license.name, url: license.url)
            end
          end

CycloneDX SBOM

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.4",
  "serialNumber": "urn:uuid:9c41660a-ecb7-431e-83c7-7f00c64610b0",
  "version": 1,
  "metadata": {
    "timestamp": "2023-03-01T12:16:45Z",
    "tools": [
      {
        "vendor": "GitLab",
        "name": "Gemnasium",
        "version": "3.11.3"
      }
    ],
    "authors": [
      {
        "name": "GitLab",
        "email": "support@gitlab.com"
      }
    ],
    "properties": [
      {
        "name": "gitlab:dependency_scanning:input_file",
        "value": "Pipfile"
      },
      {
        "name": "gitlab:dependency_scanning:package_manager",
        "value": "pipenv"
      }
    ]
  },
  "components": [
    {
      "name": "Django",
      "version": "1.11.3",
      "purl": "pkg:pypi/Django@1.11.3",
      "type": "library",
      "bom-ref": "pkg:pypi/Django@1.11.3"
    },
    {
      "name": "docutils",
      "version": "0.13.1",
      "purl": "pkg:pypi/docutils@0.13.1",
      "type": "library",
      "bom-ref": "pkg:pypi/docutils@0.13.1"
    },
    {
      "name": "pytz",
      "version": "2022.7.1",
      "purl": "pkg:pypi/pytz@2022.7.1",
      "type": "library",
      "bom-ref": "pkg:pypi/pytz@2022.7.1"
    },
    {
      "name": "requests",
      "version": "2.5.3",
      "purl": "pkg:pypi/requests@2.5.3",
      "type": "library",
      "bom-ref": "pkg:pypi/requests@2.5.3"
    }
  ]
}

Steps to reproduce

Clone python-pipenv test project into staging and edit the .gitlab-ci.yml to:

stages:
  - test

include:
  - template: Dependency-Scanning.gitlab-ci.yml

Push the updated .gitlab-ci.yml to your new forked project. After the DS job runs you should be able to see information in:

• Security & Compliance > License Compliance
• Security & Compliance > Dependency list

Example Project

An example project is python-pipenv-replace-license-finder-test-new-job

What is the current bug behavior?

What is the expected correct behavior?

When a package license is identified and can be viewed under Security & Compliance > License Compliance it should also be present under Security & Compliance > Dependency list.

Relevant logs and/or screenshots

License for django 1.11.3 is BSD-3-Clause

But license is missing in dependency list

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)
(we will only investigate if the tests are passing)

Possible fixes

Update DependencyList::Report#apply_license to normalize package names using Sbom::PackageUrl::Normalizer.

DependencyList::Report#apply_license is called from DependencyList#apply_license!.

Implementation plan

• Update Dependency to track purl_type
• Update DependencyList::Report#apply_license to compare normalized package names.
• Update the specs for DependencyList::Report#apply_license and/or the specs for DependencyList#apply_license! to cover the edge case.

See ee/spec/lib/gitlab/ci/reports/dependency_list/report_spec.rb and ee/spec/lib/gitlab/ci/parsers/security/dependency_list_spec.rb.