Report parsing error: [Schema] property '/vulnerabilities/x/name' is invalid: error_type=maxLength

Summary

When running a scan in a CI/CD pipeline, the report artifact generated gl-*-report.json contains a name field for each vulnerability which according to the schemas used for validating the report, should contain a string that does not exceed 255 characters.

This brings about an error in the pipeline's security tab: [Schema] property '/vulnerabilities/x/name' is invalid: error_type=maxLength

Steps to reproduce

Create a project with the following 2 files:

  • main.php
<?php

$url = array_filter('a' => $b')

?>
  • .gitlab-ci.yml
include:
  - template: Jobs/SAST.gitlab-ci.yml

After the pipeline completes, check the security tab of the pipeline to see the error.

Example project

https://gitlab.com/adamcohen/443628-phpcs-exceeds-vulnerabiilty-name-max-length

What is the current bug behavior?

Security scan report is not parsed properly therefore no scan results are presented in the UI.

What is the expected correct behavior?

Security scan report should be parsed properly.

Relevant logs and/or screenshots

Security_Tab_Screenshot

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Workaround

You can use jq to work around this issue by overriding the affected job. The override involves adding an after_script section to the job for example:

phpcs-security-audit-sast:
  after_script:
    - wget -O /tmp/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64
    - chmod +x /tmp/jq
    - /tmp/jq '.vulnerabilities |= map(if has("name") then .name |= .[0:255] else . end)' gl-sast-report.json > temp.json
    - /tmp/jq '.' temp.json > gl-sast-report.json
    - rm temp.json
    - rm /tmp/jq

Implementation Plan

Once Update version of report package used by comman... (#451555 - closed) • Adam Cohen • 17.3 has been completed, we can solve this issue as follows:

  1. Add a new report.TruncateTextFields() function to report/report.go:

    Click to expand report.go 
    diff --git a/report.go b/report.go
index 424ede6..b4f4d75 100644
--- a/report.go
+++ b/report.go
@@ -89,6 +89,16 @@ func (s ScannerDetails) String() string {
        return fmt.Sprintf("%s %s analyzer v%s", s.Vendor.Name, s.Name, s.Version)
 }

+// TruncateTextFields ensures that fields don't exceed the maximum allowed length
+func (r *Report) TruncateTextFields() {
+       for i := range r.Vulnerabilities {
+               name := r.Vulnerabilities[i].Name
+               if len(name) > VulnerabilityNameMaxLengthBytes {
+                       r.Vulnerabilities[i].Name = name[:VulnerabilityNameMaxLengthBytes-3] + "..."
+               }
+       }
+}
+
 // Sort the Vulnerabilities, Remediations, DependencyFiles, and DependencyFiles[].Dependencies
 func (r *Report) Sort() {
        // sort vulnerabilities by Severity, CompareKey, and Location.Dependency.Version (if available)

  2. Execute the new report.TruncateTextFields() function in command/run.go:

    Click to expand run.go 
    diff --git a/run.go b/run.go
index 36c7c74..42f9778 100644
--- a/run.go
+++ b/run.go
@@ -196,6 +196,7 @@ func Run(cfg Config) *cli.Command {
                                newReport.ApplyReportOverrides(filepath.Join(root, newReport.Config.Path), newReport.Analyzer)
                        }
                        newReport.Sort()
+                       newReport.TruncateTextFields()

                        artifactPath := filepath.Join(root, artifactName)
                        if err = cfg.Serializer(newReport, artifactPath, flagPrependPath, c.Bool("indent"), c.Bool("optimize")); err != nil {

    Execute TruncateTextFields in run command (gitlab-org/security-products/analyzers/command!55 - merged) • Adam Cohen • 17.1

    • Add unit tests for the above changes.
    • Release new version of command package with the above changes.

  3. Update all analyzers to use the new version of the command package from step 2. above:

Edited by Adam Cohen