security-products/sast does not exit >0 on found vulnerabilities?
We've noticed that sast
when run with dind
does not seem to let the CI job pick up a nonzero exit status even if 1 or more vulnerabilities are detected.
The job definition is currently:
sast:
stage: verify
image: docker:stable
variables:
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
SAST_EXCLUDED_PATHS: '.gitlab-ci.yml'
services:
- docker:stable-dind
script:
- echo "SP_VERSION='$SP_VERSION'" && echo "CI_SERVER_VERSION='$CI_SERVER_VERSION'" # Debug
- export SAST_VERSION=${SP_VERSION:-$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/')}
- echo "Using SAST version $SAST_VERSION"
- |
docker run \
--privileged \
--volume "$PWD:/code" \
--volume /var/run/docker.sock:/var/run/docker.sock \
"registry.gitlab.com/gitlab-org/security-products/sast:${SAST_VERSION}" \
/app/bin/run /code
artifacts:
when: always
paths:
- gl-sast-report.json
allow_failure: true
We expect that docker run ... /app/bin/run /code
should exit with the return code of the invoked command, e.g.:
$ docker container run ubuntu:bionic bash -c 'ls /'
bin
boot
dev
...
$ echo "$?"
0
$ docker container run ubuntu:bionic bash -c 'ls /foobar'
ls: cannot access '/foobar': No such file or directory
$ echo "$?"
2
However, it does not; it always exits 0, it seems. From the output:
$ echo "SP_VERSION='$SP_VERSION'" && echo "CI_SERVER_VERSION='$CI_SERVER_VERSION'"
SP_VERSION=''
CI_SERVER_VERSION='12.10.6'
$ export SAST_VERSION=${SP_VERSION:-$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/')}
$ echo "Using SAST version $SAST_VERSION"
Using SAST version 12-10-stable
...
+----------------------------------------------------------------------------------------+
| Severity | Tool | Location |
+----------------------------------------------------------------------------------------+
| Low | Bandit | src/python/invalid.py:16 |
| |
| Standard pseudo-random generators are not suitable for security/cryptographic |
| purposes. |
+----------------------------------------------------------------------------------------+
Uploading artifacts...
00:02
gl-sast-report.json: found 1 matching files
Uploading artifacts to coordinator... ok id=XXXXXXX responseStatus=201 Created token=XXXXXXX
Job succeeded
Here invalid.py
is a test file that purposefully uses a PRNG to be picked up by bandit
.
However, the job is listed as succeeded despite this vulnerability being detected.
We've added a hacky workaround that greps '"vulnerabilities": \[\]' gl-sast-report.json"
but would prefer not to rely on this.