Disable DS Scan on SBOM report after pipeline completion when a DS report is provided
Problem to solve
The original implementation of the DS using SBOM feature - and what shipped for the Beta - uses background processing in the rails application to execute the vulnerability scan of the uploaded SBOM reports, after pipeline completion.
However, after receiving user feedback we re-evaluated this approach and decided to re-instate the DS security report generation in the DS CI job: Spike: Provide migration path for customers tha... (#521769 - closed)
As we deliver the GA feature, we now no longer need the beta behavior, that currently duplicates the work. Considering early adoption of the feature its Beta stage, we must ensure that removing this capability will not break existing workflows.
Proposal
Since we had several discussions and diverging opinions on whether or not to keep this capability, let's review the options and compare them:
- Remove the beta behavior entirely. The only way to do dependency scanning using SBOM in the CI pipeline is via the DS analyzer and the SBOM Scan API.
- Disable the beta behavior only when it's unecessary. We keep this capability as a fallback when SBOM Scan API fails or when customers uses custom jobs.
Why should we remove it?
Removing the beta behavior avoids fragmentation of UX and implementation, which provides several advantages.
- simplified UX for customers and Gitlab team members to understand the feature
- single implementation path to maintain and debug
- single implementation for monitoring, usage metrics, caching, etc.
- opportunity to benefit from all improvements made on the generic process by other Sec groups (e.g. metrics in reports)
- clean decoupling between vulnerability scanning (AST) and results processing (SRM)
Additionally, in its current implementation, DS on SBOM scan is slowing down the processing of all security reports by doing the scan of SBOM reports while parsing all security reports. This poses serious problems of separation of concerns and error handling. As seen recently, that tight coupling is causing a lot of headache in understanding and debuging problems (#577331).
It's also mixing different domain logic into the same piece of code, where multiple teams can work at the same time, increasing risk of conflicts and regressions.
I also believe our efforts would be better placed in supporting other scanning contexts than maintaining different implementations for the same context.
Why should we keep it?
1 - It offers a safety net when the SBOM scan API fails
This is a valid argument for Limited Availability and offering some stability to customers until we gain confidence on the new implementation. However, this is not a sustainable approach to development. We have a lot of features that rely on API and background processing without a fallback. Ultimately, the feature design should provide enough reliability to sustain nominal usages, not to cover for outage situations.
2 - It can be reused for 3rd party SBOM support
3rd party SBOM support could be provided via the DS analyzer too. Similarly to situations where a preceding build job provides a scannable artifact (lockfile or graph file), the DS analyzer could take a custom SBOM as an input document. At least for use cases that fit in the CI pipeline workflow. If we want to start supporting SBOM scan in different scanning contexts, the beta solution would not be suitable anyway, as it is tightly coupled with report ingestion logic from CI pipeline workflow.
3 - Some customers might already be using it
Customers who onboarded early on the DS analyzer and uses a custom CI job implementation that doesn't declare the DS security report artifact will indeed be impacted. In such situation, the SBOM scan would happen in the CI job but scan results would not be uploaded as report artifact and no vulnerability scan would happen on SBOM ingestion.
Mitigations:
- Identify such usage (can we?), track it in metrics and warn user
- Offer an opt-in toggle to re-enable it if a customer depends on it (even temporarily until they adjust their CI configuration)
- Ultimately, and while unfortunate, Beta feature can face breaking changes: "breaking changes can occur outside of major releases or with less notice than for generally available features."