Skip to content

Make CS QA test more specific

Adam Cohen requested to merge update-cs-templates into master

The previous version of the container scanning qa test compared the unapproved array of vulnerabilities as well as the keys of the vulnerabilities to ensure the actual report matched the expected report.

This MR improves the container scanning qa test by comparing the complete JSON report, including both keys and values. It sorts both the actual report and the expected report first by the vulnerability name and then by the featurename. It's important to sort by both of these fields since there can exist duplicate vulnerability entries which will prevent the reports from matching as in this failed job:

       "description": "urllib in Python 2.x through 2.7.16 supports the local_file: scheme, which makes it easier for remote attackers to bypass protection mechanisms that blacklist file: URIs, as demonstrated by triggering a urllib.urlopen('local_file:///etc/passwd') call.",
       "namespace": "debian:9",
       "vulnerability": "CVE-2019-9948",
-      "featureversion": "3.5.3-1+deb9u1",
-      "featurename": "python3.5"
+      "featureversion": "2.7.13-2+deb9u3",
+      "featurename": "python2.7"
     },
     {
       "severity": "Medium",
@@ -6000,8 +6000,8 @@
       "description": "urllib in Python 2.x through 2.7.16 supports the local_file: scheme, which makes it easier for remote attackers to bypass protection mechanisms that blacklist file: URIs, as demonstrated by triggering a urllib.urlopen('local_file:///etc/passwd') call.",
       "namespace": "debian:9",
       "vulnerability": "CVE-2019-9948",
-      "featureversion": "2.7.13-2+deb9u3",
-      "featurename": "python2.7"
+      "featureversion": "3.5.3-1+deb9u1",
+      "featurename": "python3.5"
     }
   ],

It also ignores the image field in the report, since this contains a variable value which changes based on the name of the docker image being scanned.

Edited by Adam Cohen

Merge request reports