CycloneDX SBOM export for a CI pipeline
<!-- This template is a great use for issues that are feature::additions or technical tasks for larger issues.--> ### Problem to Solve Some customers currently rely on a license artifact that results from using the License Compliance template, which is being removed. Customers use this artifact to create custom automation jobs that are tied to a component's license. By providing an API that produces SBOM results from a given pipeline, customers may continue to automate other processes that related to licenses detected in their pipeline. Because this is pipeline-specific, customers may also benefit from getting the SBOM for a specific git tag (release). Given the current industry focus on Supply Chain Management and providing the Software Bill of Materials (SBOM), I would like to propose a feature request. Add the ability in GitLab to be able to export SBOMs uploaded by CI jobs into a single [CycloneDX](https://cyclonedx.org/) SBOM. Of note, CycloneDX has now joined the [OWASP Foundation](https://cyclonedx.org/news/cyclonedx-joins-owasp-foundation/). ### What does success look like, and how can we measure that? Users can download a single SBOM that lists all the components detected in a CI pipeline. It includes both application-level dependencies and system-level dependencies. ### Proposal SBOM is exportable via API in CycloneDX format for a given pipeline. Feature is behind a FF. https://gitlab.com/gitlab-org/gitlab/-/issues/333463#note_1420680496 #### Not in Scope for Alpha release - Adding any additional data that is not currently available in our SBOMs today - license information for system dependencies (because https://gitlab.com/gitlab-org/gitlab/-/issues/370013 is required) - local dependencies - accessible from the UI - Group-level SBOM export (see https://gitlab.com/gitlab-org/gitlab/-/issues/407453) ### Documentation **TBD:** Document feature. Provide scripts. See https://gitlab.com/gitlab-org/gitlab/-/issues/333463#note_1496971495 Document the new API endpoint as alpha (in case we need to tweak it before 17.0). See https://gitlab.com/gitlab-org/gitlab/-/issues/333463#note_1420680496 ### Implementation Plan Following implementation can be broken down into 4 separate MRs. _[MR-1](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/127591):_ - [x] Add `pipeline_id` and `export_type` fields to `dependency_list_exports` table. - [x] Create index `[:pipeline_id]` - [x] Add `:pipeline_id` [exportable](https://gitlab.com/gitlab-org/gitlab/-/blob/master/ee/app/models/dependencies/dependency_list_export.rb#L49) type. `pipeline_id` exportable type can co-exist with the project for this implementation. _[MR-2](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/128872):_ - [x] Add `MergeSbomService`. `MergeSbomService` should be responsible for merging sbom [reports](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/reports/sbom/reports.rb#L11). _[MR-3](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/128974):_ - [x] Create a feature flag. Feature Flag issue: https://gitlab.com/gitlab-org/gitlab/-/issues/421513 - [x] Add `create_sbom_export` method to generate the merged `sbom_export` file. - [x] This should also validate the schema of the merged sbom file. [Gitlab::Ci::Parsers::Sbom::Validators::CyclonedxSchemaValidator](https://gitlab.com/gitlab-org/gitlab/-/blob/master/lib/gitlab/ci/parsers/sbom/validators/cyclonedx_schema_validator.rb) should be used for validation. - [x] Add the following API endpoint to generate and download the merged sbom report: - - [x] `pipeline/:id/dependency_list_exports` endpoint to generate the SbomExport in the background using sidekiq. _[MR 4](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/130024):_ - [x] Document the feature along with the example script. Provide an example to upload the file as job artifacts. Note: The SBOM export would be uploaded as a normal artifact (like in the [script you've shared](https://gitlab.com/gitlab-org/gitlab/-/issues/333463#note_1501595488)), and not as a `cyclonedx` report, otherwise the backend would end up ingesting duplicate SBOM components. - [x] Enable the feature flag. _MR 5:_ - Add e2e tests if required. ### Verification steps **TBD** <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION --> *This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.* <!-- triage-serverless v3 PLEASE DO NOT REMOVE THIS SECTION -->
issue