## Overview Enable the ingestion and processing SPDX license expressions for supported package managers in the Package Metadata DB (PMDB) infrastructure. The epic is part of the work needed to add https://gitlab.com/groups/gitlab-org/-/epics/16801+s ### Problem to solve SPDX license expressions cannot be supported in the GitLab instance until that data is provided via PMDB. Today PMDB does not handle spdx expressions and treats them as unknown when encountered. We will need a v3 in order to ensure that Rails instances will be able to import SPDX LE data and know how to parse them. If we export these data in v2 then older GitLab versions will sync SPDX LE data but won't be able to store the data in the database. Later when they upgrade we would need a migration script to re-sync. This problem was present also when we introduced cvss_v4. ### Scope 1. Compound expressions: AND, OR, and parenthesised groupings (e.g. MIT OR (Apache-2.0 AND GPL-2.0-only)) 2. Exception qualifiers: WITH followed by a valid SPDX exception identifier (e.g. GPL-2.0-only WITH Classpath-exception-2.0) 3. Or-later / the + suffix (e.g. GPL-2.0+) 4. License references: LicenseRef-, AdditionRef-, and DocumentRef- See https://spdx.github.io/spdx-spec/v3.0.1/annexes/spdx-license-expressions/ ### Out of scope #### Dereferencing custom license references In thie spic, we will treat `LicenseRef-*`, `DocumentRef-*`, and `AdditionRef-*` tokens as opaque strings within expressions. We will store and export the expression as-is (e.g. `Apache-2.0 AND LicenseRef-Proprietary`) without resolving or storing the license text that the reference points to. ### Proposed solution Implement the complete PMDB pipeline changes to support SPDX expressions: #### Database Schema Updates - Add `spdx_expression` columns to \<component\>\_license table - Maintain backward compatibility with existing license functionality #### License Feeder Updates - Add necessary support for each package manager feeder type (TBC) #### License Processor Enhancements - Implement SPDX expression detection and validation logic - Store expressions in the database #### License Exporter Updates - Modify export logic to conditionally return SPDX license name or expression - Maintain existing data format for downstream consumers ### Success Criteria - [ ] PMDB can successfully ingest and store SPDX license expressions for supported package managers - [ ] License exporter can export license expressions while remaining compatible with `version_format: v2` package_metadata ### References - [Proof of concept parser](https://gitlab.com/hacks4oats/spdx-license-evaluator) - [Parent](https://gitlab.com/groups/gitlab-org/-/epics/16801) ### Target Metric/s * Improved detection of licenses resulting in an increased % of licenses accurately detected, which can lead to an increase in adoption of SCA. _(Will update with data via comment section)_