Report parsing error: [Schema] property '/vulnerabilities/x/name' is invalid: error_type=maxLength
Summary
When running a scan in a CI/CD pipeline, the report artifact generated gl-*-report.json
contains a name field for each vulnerability which according to the schemas used for validating the report, should contain a string that does not exceed 255 characters.
This brings about an error in the pipeline's security tab: [Schema] property '/vulnerabilities/x/name' is invalid: error_type=maxLength
Steps to reproduce
Create a project with the following 2 files:
- main.php
<?php
$url = array_filter('a' => $b')
?>
- .gitlab-ci.yml
include:
- template: Jobs/SAST.gitlab-ci.yml
After the pipeline completes, check the security tab of the pipeline to see the error.
Example project
https://gitlab.com/adamcohen/443628-phpcs-exceeds-vulnerabiilty-name-max-length
What is the current bug behavior?
Security scan report is not parsed properly therefore no scan results are presented in the UI.
What is the expected correct behavior?
Security scan report should be parsed properly.
Relevant logs and/or screenshots
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Workaround
You can use jq
to work around this issue by overriding the affected job. The override involves adding an after_script
section to the job for example:
phpcs-security-audit-sast:
after_script:
- wget -O /tmp/jq https://github.com/jqlang/jq/releases/download/jq-1.7.1/jq-linux-amd64
- chmod +x /tmp/jq
- /tmp/jq '.vulnerabilities |= map(if has("name") then .name |= .[0:255] else . end)' gl-sast-report.json > temp.json
- /tmp/jq '.' temp.json > gl-sast-report.json
- rm temp.json
- rm /tmp/jq
Implementation Plan
Once Update version of report package used by comman... (#451555 - closed) • Adam Cohen • 17.3 has been completed, we can solve this issue as follows:
-
Add a new report.TruncateTextFields()
function to report/report.go:Click to expand
report.go
diff --git a/report.go b/report.go index 424ede6..b4f4d75 100644 --- a/report.go +++ b/report.go @@ -89,6 +89,16 @@ func (s ScannerDetails) String() string { return fmt.Sprintf("%s %s analyzer v%s", s.Vendor.Name, s.Name, s.Version) } +// TruncateTextFields ensures that fields don't exceed the maximum allowed length +func (r *Report) TruncateTextFields() { + for i := range r.Vulnerabilities { + name := r.Vulnerabilities[i].Name + if len(name) > VulnerabilityNameMaxLengthBytes { + r.Vulnerabilities[i].Name = name[:VulnerabilityNameMaxLengthBytes-3] + "..." + } + } +} + // Sort the Vulnerabilities, Remediations, DependencyFiles, and DependencyFiles[].Dependencies func (r *Report) Sort() { // sort vulnerabilities by Severity, CompareKey, and Location.Dependency.Version (if available)
-
Add unit tests for the new TruncateTextFields
functionUse Add TruncateTextFields function (gitlab-org/security-products/analyzers/report!79 - merged) • Adam Cohen as a starting point.
-
Release new version of report
package with the above changes.
-
-
Execute the new report.TruncateTextFields()
function in command/run.go:Click to expand
run.go
diff --git a/run.go b/run.go index 36c7c74..42f9778 100644 --- a/run.go +++ b/run.go @@ -196,6 +196,7 @@ func Run(cfg Config) *cli.Command { newReport.ApplyReportOverrides(filepath.Join(root, newReport.Config.Path), newReport.Analyzer) } newReport.Sort() + newReport.TruncateTextFields() artifactPath := filepath.Join(root, artifactName) if err = cfg.Serializer(newReport, artifactPath, flagPrependPath, c.Bool("indent"), c.Bool("optimize")); err != nil {
-
Add unit tests for the above changes. -
Release new version of command
package with the above changes.
-
-
Update all analyzers to use the new version of the command
package from step2.
above:-
SAST -
secrets -
security-code-scan -
spotbugs -
semgrep -
sobelow -
pmd-apex -
kubesec -
kics -
phpcs-security-audit(analyzer has been removed) -
mobsf(analyzer has been removed) -
flawfinder(analyzer has been removed) -
nodejs-scan(analyzer has been removed) -
brakeman(analyzer has been removed)
-
-
DAST -
gitlab-cov-fuzz-src -
browserker(does not use the command or report packages)
-
-
Vulnerability Research -
npm-audit(project is archived)
-
-
Composition Analysis
-