<!-- The first section "Release notes" is required if you want to have your release post blog MR auto generated. Currently in BETA, details on the **release post item generator** can be found in the handbook: https://about.gitlab.com/handbook/marketing/blog/release-posts/#release-post-item-generator and this video: https://www.youtube.com/watch?v=rfn9ebgTwKg. The next four sections: "Problem to solve", "Intended users", "User experience goal", and "Proposal", are strongly recommended in your first draft, while the rest of the sections can be filled out during the problem validation or breakdown phase. However, keep in mind that providing complete and relevant information early helps our product team validate the problem and start working on a solution. --> ### Release notes <!-- What is the problem and solution you're proposing? This content sets the overall vision for the feature and serves as the release notes that will populate in various places, including the [release post blog](https://about.gitlab.com/releases/categories/releases/) and [Gitlab project releases](https://gitlab.com/gitlab-org/gitlab/-/releases). " --> ### Problem to solve <!-- What problem do we solve? Try to define the who/what/why of the opportunity as a user story. For example, "As a (who), I want (what), so I can (why/value)." --> To support [replacing license finder](https://gitlab.com/groups/gitlab-org/-/epics/8072), we need to have a central, uniformly formatted collection of data that maps license data to components. ### Intended users <!-- Who will use this feature? If known, include any of the following: types of users (e.g. Developer), personas, or specific company roles (e.g. Release Manager). It's okay to write "Unknown" and fill this field in later. Personas are described at https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/ * [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager) * [Parker (Product Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#parker-product-manager) * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Presley (Product Designer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#presley-product-designer) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) * [Priyanka (Platform Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#priyanka-platform-engineer) * [Sidney (Systems Administrator)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sidney-systems-administrator) * [Sam (Security Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sam-security-analyst) * [Rachel (Release Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#rachel-release-manager) * [Alex (Security Operations Engineer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#alex-security-operations-engineer) * [Simone (Software Engineer in Test)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#simone-software-engineer-in-test) * [Allison (Application Ops)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#allison-application-ops) * [Ingrid (Infrastructure Operator)](https://about.gitlab.com/handbook/product/personas/#ingrid-infrastructure-operator) * [Dakota (Application Development Director)](https://about.gitlab.com/handbook/product/personas/#dakota-application-development-director) * [Dana (Data Analyst)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#dana-data-analyst) * [Eddie (Content Editor)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#eddie-content-editor) --> * [Cameron (Compliance Manager)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#cameron-compliance-manager) * [Delaney (Development Team Lead)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#delaney-development-team-lead) * [Sasha (Software Developer)](https://about.gitlab.com/handbook/marketing/product-marketing/roles-personas/#sasha-software-developer) ### Proposal <!-- How are we going to solve the problem? Try to include the user journey! https://about.gitlab.com/handbook/journeys/#user-journey --> 1. A new repository will be created, similar to https://gitlab.com/gitlab-org/security-products/gemnasium-db called "GitLab License Database". 1. The GitLab License Database will be [licensed similarly](https://gitlab.com/gitlab-org/security-products/gemnasium-db/-/blob/master/LICENSE.md) to the GitLab Advisory Database. The exact license will need to be approved by legal. 1. The stored format will be JSON and will include the following information at a minimum: 1. `date` 1. `registry` (should be either) 1. `component_name` 1. `component_version` 1. `license_name` (this can be an array if there are multiple licenses involved) 1. A regular daily job will be created to read license data from the following registries, import the data, and store it in the GitLab License Database in a consistent format: | language | packages | API docs | docs | | -------- | -------- | -------- | ------ | | JavaScript | npm, yarn | [API docs](https://github.com/npm/registry/blob/master/docs/REGISTRY-API.md) | See https://gitlab.com/gitlab-org/secure/vulnerability-research/awesomesauce/-/issues/90#note_1045562846 | | Ruby | Bundler | [API docs](https://guides.rubygems.org/rubygems-org-api/) | API calls are authenticated so we likely need an account. A listing of all packages can be obtained at https://rubygems.org/versions using the [Compact Index](https://andre.arko.net/2014/03/28/the-new-rubygems-index-format/) as demonstrated in the [adapter implementation](https://gitlab.com/gitlab-org/security-products/license-db/rubygems-adapter/-/blob/main/main.go#L47). | | PHP | Composer | [API docs](https://getcomposer.org/doc/05-repositories.md) | A listing of all packages can be obtained at https://packagist.org/packages/list.json?filter=* and license data can be obtained by querying for individual packages via https://repo.packagist.org/p2/{{package_name}}.json (for example https://repo.packagist.org/p2/phpcommon/comparison.json) | | C, C++ | Conan | https://github.com/conan-io/conan-center-index.git | A listing of all package can be obtained by listing all files in the [`listing` subdir](https://github.com/conan-io/conan-center-index/tree/master/recipes) as demonstrated in the [adapter implementation](https://gitlab.com/gitlab-org/security-products/license-db/conan-adapter/-/blob/main/main.go#L70). | | Go | Go | license information may be detected from the source repositories of the go module dependencies | See https://index.golang.org | | Java | Gradle, Maven | [Sonatype](https://central.sonatype.org/search/rest-api-guide/), [MVNRepository](https://mvnrepository.com/) | | | .NET, C# | NuGet | [API docs](https://docs.microsoft.com/en-us/nuget/api/overview) | Packages can be listed by means of the `https://api.nuget.org/v3/catalog0/index.json` API as implemented in [the Nuget adapter](https://gitlab.com/gitlab-org/security-products/license-db/nuget-adapter/-/blob/main/main.go#L247). | | Python | setuptools, Pip, Pipenv, Poetry | [API docs](https://warehouse.pypa.io/api-reference/) | Package index is accessible through the pypi `/simple` API endpoint [as implemented in our adapter](https://gitlab.com/gitlab-org/security-products/license-db/pypi-adapter/-/blob/main/main.go#L71). | ### Further details <!-- Include use cases, benefits, goals, or any other details that will help us understand the problem better. --> This is intended to be somewhat similar to https://gitlab.com/gitlab-org/security-products/gemnasium-db only it will be maintained by gitlab~10690742 and will store license data instead of vulnerability data. ### Available Tier <!-- This section should be used for setting the appropriate tier that this feature will belong to. Pricing can be found here: https://about.gitlab.com/pricing/ * Free * Premium/Silver * Ultimate/Gold --> gitlab~3207279 <!-- Label reminders - you should have one of each of the following labels. Use the following resources to find the appropriate labels: - https://gitlab.com/gitlab-org/gitlab/-/labels - https://about.gitlab.com/handbook/product/categories/features/ --> <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> <!-- start-discoto-summary --> ## Auto-Summary :robot: <details> <summary>Discoto Usage</summary> --- > **Points** > > Discussion points are declared by headings, list items, and single > lines that start with the text (case-insensitive) `point:`. For > example, the following are all valid points: > > * `#### POINT: This is a point` > * `* point: This is a point` > * `+ Point: This is a point` > * `- pOINT: This is a point` > * `point: This is a **point**` > > Note that any markdown used in the point text will also be propagated > into the topic summaries. > > **Topics** > > Topics can be stand-alone and contained within an issuable (epic, > issue, MR), or can be inline. > > Inline topics are defined by creating a new thread (discussion) > where the first line of the first comment is a heading that starts > with (case-insensitive) `topic:`. For example, the following are all > valid topics: > > * `# Topic: Inline discussion topic 1` > * `## TOPIC: **{+A Green, bolded topic+}**` > * `### tOpIc: Another topic` > > **Quick Actions** > > | Action | Description | > |-------------------------------|---------------------------------------------------------| > | `/discuss sub-topic TITLE` | Create an issue for a sub-topic. Does not work in epics | > | `/discuss link ISSUABLE-LINK` | Link an issuable as a child of this discussion | > --- </details> Last updated by [this job](https://gitlab.com/gitlab-org/secure/pocs/discoto-runner/-/jobs/2798419288) * **TOPIC** File-based database (Git repository) https://gitlab.com/groups/gitlab-org/-/epics/8492#note_1044959093 * **TOPIC** Maintenance effort https://gitlab.com/groups/gitlab-org/-/epics/8492#note_1044959098 * **TOPIC** Alternative solution -- lazy license database creation https://gitlab.com/groups/gitlab-org/-/epics/8492#note_1044959104 * **TOPIC** Alternative solution -- using a 3rd party license database https://gitlab.com/groups/gitlab-org/-/epics/8492#note_1044959105 * **TOPIC** Architecture https://gitlab.com/groups/gitlab-org/-/epics/8492#note_1046645314 <!-- end-discoto-summary --> <!-- start-discoto-topic-settings --><details> <summary>Discoto Settings</summary> <br/> ```yaml --- summary: max_items: -1 sort_by: created sort_direction: ascending ``` See the [settings schema](https://gitlab.com/gitlab-org/secure/pocs/discussion-automation#settings-schema) for details. </details> <!-- end-discoto-topic-settings -->