Skip to content

Fix v2 default licenses

Igor Frenkel requested to merge 418114-fix-v2-license-ingestion into master

What does this MR do and why?

Fix mutating concat when fetching license data from data objects. Mutating concat causes all licenses to be recorded as default_licenses which results in incorrect results.

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

Before After

How to set up and validate locally

  1. Create a vendor/package_metadata/licenses/v2/pypi/0/0.ndjson file with contents of one line as above:

    {"name":"requests","lowest_version":"2.3.0","highest_version":"2.31.0","default_licenses":["Apache-1.0","Apache-2.0"],"other_licenses":[{"licenses":["ISC"],"versions":["0.10.2"]},{"licenses":["Apache-1.0"],"versions":["1.0.0"]},{"licenses":["unknown"],"versions":["0.13.2","0.13.5"]},{"licenses":["MIT"],"versions":["0.0.1"]}]}
  2. bundle exec rails console

  3. Run sync service

    # optionally disable other purl_types if you have data under vendor/package_metadata/licenses already
    lease = Gitlab::ExclusiveLease.new("foo", timeout: 5.minutes)
    lease.try_obtain
    PackageMetadata::SyncService.execute(data_type: 'licenses', lease: lease)
  4. Check default licenses

    license_ids = PackageMetadata::Package.find_by(purl_type: 'pypi', name: 'requests').licenses[0]
    PackageMetadata::License.where(id: license_ids).pluck(:spdx_identifier)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #418114 (closed)

Merge request reports