Skip to content

Fix CS QA script bug

Adam Cohen requested to merge fix-cs-qa-test into master

While updating the container scanning test project, I noticed the following qa-container_scanning job was passing, even though jq was returning false:

$ jq --argfile actual ./$REPORT --argfile expected ./qa/expect/$REPORT -n '($actual | { vulnerabilities, unapproved } | .vulnerabilities |= sort_by(.vulnerability, .featurename)) as $actual | ($expected | { vulnerabilities, unapproved } | .vulnerabilities |= sort_by(.vulnerability, .featurename)) as $expected | $actual == $expected'
false
Job succeeded

This bug was introduced by my change to the jq command in Make CS QA test more specific due to the removal of the -e / --exit-status flag.

This MR fixes the issue by re-adding the -e flag, which will cause jq to return an error exit status if the comparison of container scanning reports returns false.

Once this MR has been merged, the qa-container_scanning job for projects such as ruby-bundler and js-npm will fail and will need to be fixed, which I'll create MRs to address.

Merge request reports