## Purpose Identify all analyzers currently outputting invalid JSON according to the latest relevant security report schemas. Establish and plan a timeline for bringing all affected analyzers into compliance with a target of %"15.0" at the latest. This will include identifying and reaching out to all 3rd-party tool vendors, where possible. ## Background Our published [security report schemas](https://docs.gitlab.com/ee/development/integrations/secure.html#report) are supposed to ensure proper communication of data between security scanning tools running in pipeline jobs and our vulnerability management capabilities. These schemas are used by GitLab-provided scanners as well as many integrated 3rd-party tools. Historically, there has not been proper validation and enforcement of these schemas. This led to silent parsing errors not exposed to the user and in some cases bad or incomplete data making it into findings/vulnerability records. To remedy this, we plan to start [validating all artifacts](https://gitlab.com/gitlab-org/gitlab/-/issues/335789) and [stop ingesting invalid reports](https://gitlab.com/gitlab-org/gitlab/-/issues/327238). As this will effectively break non-compliant analyzers, we need to ensure the work done to make all of ours complaint is completed prior to turning on schema enforcement and rejection of invalid reports. ## Decisions 1. From GitLab %14.7 (was the be %"14.6" but deprecation notice didn't make the release), security report schema versions under 14.0.0 are deprecated. 1. From GitLab %14.7, all security reports must use [security report schema version 14 and above](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/tree/v14.0.5/dist). 1. From GitLab %14.10 at the latest, and sooner if possible, any non-compliant reports will cause a [warning](https://gitlab.com/gitlab-org/gitlab/-/issues/335789#note_672853791) to be displayed to users in the Vulnerability Report. 1. From GitLab release %15.0 the following reports will no longer be ingested: 1. Reports that use an unsupported schema version. 1. Reports that do not pass validation against the schema version declared in the report. - ~backend DRI: @Quintasan - ~frontend DRI: TBC @nmccorrison ## Scenarios | Scenario | GitLab Version >= 15.0? | Version present? | Version supported? | Version deprecated? | Report validates? | Result | Status | | ------ | ------ | ------ | ------ | ------ | ------ | ------ | ------ | | `supported security scan report passes validation` | n/a | :white_check_mark: yes | :white_check_mark: yes | :white_check_mark: no | :white_check_mark: yes | :white_check_mark: ingest | implemented | | `deprecated security scan report passes validation` | n/a | :white_check_mark: yes | :white_check_mark: yes | :x: yes | :white_check_mark: yes | :warning: ingest | implemented | | `supported security scan report fails validation pre 15.0` | :x: no | :white_check_mark: yes | :white_check_mark: yes | :white_check_mark: no | :x: no | :warning: ingest | implemented | | `deprecated security scan report fails validation pre 15.0` | :x: no | :white_check_mark: yes | :white_check_mark: yes | :x: yes | :x: no | :warning: ingest | implemented | | `unsupported security scan report is submitted pre 15.0 ` | :x: no | :white_check_mark: yes | :x: no | n/a | n/a | :warning: ingest | implemented | | `unversioned security scan report is submitted pre 15.0 ` | :x: no | :x: no | n/a | n/a | n/a | :warning: ingest | implemented | | `supported security scan report fails validation` | :white_check_mark: yes | :white_check_mark: yes | :white_check_mark: yes | :white_check_mark: no | :x: no | :no_entry: reject | implemented | | `deprecated security scan report fails validation` | :white_check_mark: yes | :white_check_mark: yes | :white_check_mark: yes | :x: yes | :x: no | :no_entry: reject | implemented | | `unsupported security scan report is submitted ` | :white_check_mark: yes | :white_check_mark: yes | :x: no | n/a | n/a | :no_entry: reject | implemented | | `unversioned security scan report is submitted ` | :white_check_mark: yes | :x: no | n/a | n/a | n/a | :no_entry: reject | implemented | | `security scan report can't be parsed ` | :white_check_mark: yes | :question: invalid JSON | n/a | n/a | n/a | :no_entry: reject | implemented | ## Action Items - [x] Announce deprecation/breaking change planned for %"15.0": https://gitlab.com/gitlab-org/gitlab/-/merge_requests/74218 - [x] Clarify which [schema releases](https://gitlab.com/gitlab-org/security-products/security-report-schemas/-/releases) will be supported from %15.0. - [x] Identify all affected analyzers - [x] Create a child issue for each to track progress: - [x] [DAST API](https://gitlab.com/gitlab-org/gitlab/-/issues/330368) - [x] [Coverage Fuzzing](https://gitlab.com/gitlab-org/gitlab/-/issues/323703) - [x] [Container Scanning](https://gitlab.com/gitlab-org/gitlab/-/issues/344612) - [x] Notify any affected integration partners