Replace hard-coded list of software licenses

Summary

I would like to replace the hard coded list of licenses found here with an API endpoint to return a list of software licenses or drive this list from SPDX licenses.json.

Improvements

The main benefit of this improvement is that it would provide a more comprehensive and up to date list of software licenses to choose from. This list would include SPDX identifiers that has a higher chance of detection in our software license policy scans. This would also reduce the need for end users to manually enter license names that we do not have a matching SPDX identifer for.

Option 1: GET /api/vX/software_licenses.json

This would return a list software licenses stored in the software_licenses table that has a valid spdx_identifier. Any software_licenses with a spdx_identifier would not be included. Once this MR is merged the only software licenses in the database with an unknown spdx_identifier will be the ones that were user entered.

Option 2: GET https://spdx.org/licenses/licenses.json

I don't think we want to fetch the licenses.json directly from spdx.org as CSP could block us in the future and it's not ideal to depend on a third party resource without known SLA. We could look into the option of hosting the licenses.json file as a static asset served from public/ then fetch the resource on demand. This would allow us to use the same license identifier found in the SPDX catalog and improve the efficacy of our license detection by depending on identifiers that aren't likely to change. Today we depend on the license name, that we have observed can have multiple representations for the same logical license.

Risks

An addition API call will need to be made to fetch the list of software licenses to choose from in the software policy configuration page.

Involved components

• https://gitlab.com/gitlab-org/gitlab/blob/4d8ed0b13d357de6b8e8e1e50aaf9617287de727/ee/app/assets/javascripts/vue_shared/license_management/constants.js#L7-33

Optional: Intended side effects

Our list of software licenses will be up to date and match the identifiers used by the greater software community. https://spdx.org

This idea was first discussed here.