Dependency Scanning schema is missing required `package` object

Summary

The Dependency Scanning (DS) report schema validates reports that the gitlab backend can't process because dependency_files[].dependencies[].package is missing, even though the schema doesn't require that field. See dependency-scanning-report-format.json.

For example, the Dependency List page displays an error after ingesting a DS report that has no package field:

Error fetching the dependency list. Please check your network connection and try again.

This is caused by a 500 error in the REST endpoint used by the page (e.g. https://gitlab.com/thiagocsf-group/demo/cs-with-deps/-/dependencies.json).

Further details

The error can be seen in Sentry:

app/models/ci/build.rb in block (2 levels) in each_report at line 1141

        def each_report(report_types)
          job_artifacts_for_types(report_types).each do |report_artifact|
            report_artifact.each_blob do |blob|
              yield report_artifact.file_type, blob, report_artifact
            end
          end
        end

The code depends on package object to have a name attribute here: ee/lib/gitlab/ci/parsers/security/formatters/dependency_list.rb:98.

But since there is no package object it’s raising an exception. The schema validates because the package is not a required attribute for the items of the dependencies list.

Both the dependency_files array and the location object of src/dependency-scanning-report-format.json use the dependency object defined in src/security-report-format.json, and this object has no required property. Also, dependency has a package field, but the definition of package has no required field either.

Steps to reproduce

  1. Create a security report that passes validation for dependency-scanning-report-format.json. Here's an example.
  2. Submit the report as a dependency_scanning report (example)
  3. Go to Dependency List under Security & Compliance (still broken example; fixed example - the latter works because the latest pipeline ran with the fixed report)

Example Project

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

image

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Implementation plan

PTAL at this started MR as it could be reused instead of starting from scratch: gitlab-org/security-products/security-report-schemas!114 (merged)

Update the Dependency Scanning JSON schema and declare new required fields:

  1. List package and version as required properties of dependency.
  2. List name as a required property of package.
  3. Release a new MODEL because the new JSON has new required field. See https://gitlab.com/gitlab-org/security-products/security-report-schemas/#classifying-changes This will be done with #363134 (closed)

There's no need to modify the Dependency Scanning analyzers since they all already produced these fields.

/cc @NicoleSchwartz @gonzoyumo

Edited by Olivier Gonzalez