SAST analyzers don't propagate stderr of underlying tools
Summary
The SAST analyzers don't propagate the standard error of the underlying tools. This makes debugging really hard.
Steps to reproduce
Run some Docker-based analyzer on a project that is either broken or not compatible. The analyzer fails but the output doesn't contain what's reported by the underlying tool.
Example Project
Any project that's incompatible with the analyzer can be used to reproduce this issue. The tool the analyzer is based on will certainly report some error on stderr.
What is the current bug behavior?
The analyzer output doesn't contain what the underlying tool writes in stderr:
$ docker run -ti --rm --volume $PWD:/tmp/app --env CI_PROJECT_DIR=/tmp/app registry.gitlab.com/gitlab-org/security-products/analyzers/brakeman:11-0-stable
Found project in /tmp/app
2018/06/13 06:41:23 open /tmp/brakeman.json: no such file or directory
What is the expected correct behavior?
It should report some error when running brakeman on an incompatible project:
$ docker run -ti --rm --volume $PWD:/tmp/app --env CI_PROJECT_DIR=/tmp/app registry.gitlab.com/gitlab-org/security-products/analyzers/brakeman:11-0-stable /bin/bash
root@337e7ebcb5f0:/# cd /tmp/app
root@337e7ebcb5f0:/tmp/app# brakeman
NOTE: Gem.gunzip is deprecated; use Gem::Util.gunzip instead. It will be removed on or after 2018-12-01.
Gem.gunzip called from /usr/local/bundle/gems/brakeman-4.2.1/bundle/ruby/2.5.0/gems/unicode-display_width-1.3.0/lib/unicode/display_width/index.rb:5.
Loading scanner...
Please supply the path to a Rails application (looking in /tmp/app).
Here the interesting line is:
Please supply the path to a Rails application (looking in /tmp/app).
Possible fixes
Go-based analyzers redirect stderr to cli.App.ErrWriter
(see cli.App
) but that doesn't work for some reasons. Redirecting the output to cli.App.Writer
works. Actually, this is why stdout is correctly propagated.
Even though it doesn't prevent SAST from working properly, I recommend fixing this problem in the next release to make debugging a lot easier.