Extend Security reports format to include API version
Problem to solve
The security report JSON output evolves over time and while we often can simply enrich it, some deprecated keys are useless and may be dropped in a Major release. It is also difficult for the consumer to extrapolate an implicit versioning based on the existence of some keys and decide how to parse it from this guess.
So to make it clear for the consumer of the report which format it is dealing with, we should explicitly version the output format and include that version as part of the report itself.
Let's start with SAST and DS for now and then apply to other Security Products as we spread usage of the orchestrator + analyzers architecture.
IMPORTANT: Doing this is a breaking change so we'd better wait to have completed the issues related to versioning of SAST and its analyzers: https://gitlab.com/gitlab-org/gitlab-ee/issues/7139, https://gitlab.com/gitlab-org/gitlab-ee/issues/6797, https://gitlab.com/gitlab-org/gitlab-ee/issues/8000
Further details
The report JSON output is the important synchronization point between GitLab backend (or external consumer) and the Security Products (like SAST) and their underlying analyzers. By providing the version of the format we make the consumer aware of how to parse the vulnerabilities at all time, making it way easier to handle backward compatibility while still being able to move forward.
Proposal
Instead of having security report being just an array of vulnerabilities, we should structure it this way:
{
"version": 1,
"vulnerabilities": [...]
}
To do so we should probably set the format version within the Analyzers' Common library.
We are currently at version 1.2
(or 1.3
, see last point):
-
1.0
: initial release when introducing common library -
1.1
: enrichment of the report with more metadata (keeping renamed/deprecated properties) -
1.2
: addscanner
and deprecatetool
-
1.3
: adddependency
sub object tolocation
(Ongoing, see #8654 (closed))
But with the move to this new format we're breaking backward compatibility so we need to bump to 2.0
.
What does success look like, and how can we measure that?
Security Report JSON output contains its version
Links / references
https://gitlab.com/gitlab-org/gitlab-ee/issues/6797#note_101755951