SAST Spotbugs fails with 'exit status 1' (killed; memory issue)
Summary
A customer experienced SAST Spotbugs failing, and this issue covers off the issues that arose, and the improvements they proposed in the ticket (internal link).
The customer's experience when SpotBugs fails is as follows:
Project built.
exit status 1
Error: SpotBugs analysis failed for /tmp/app: exit status 1
Uploading artifacts...
WARNING: gl-sast-report.json: no matching files
ERROR: No files to upload
From the ticket (also comment here)
- You swallow exceptions, that's nasty.
- So by adding more information if something fails i was able to discover spotbugs process is silently killed. See: https://gitlab.com/widerin/spotbugs/-/commit/be2205dd136e1205dd22cbb15bd0e9e3ac99ddb6
- The problem seems to be the JVM configuration, as containers on our CI infrastructure usually get enough memory.
- I've seen by default you set JAVA_OPTS (-Xmx1900M). IMHO, by-default this should be not set. When people still use Java 8 with lack of container support, they might want to configure this or you should mention this in docs. On Java 11 Container-support is enabled by default. We could discuss setting a relative max heap size here, e.g. -XX:MaxRAMPercentage=90
workaround
If other customers experience spotbugs failing in this way, it maybe related to memory. Depending on the JVM version you're using, Override JAVA_OPTS
in SAST job directly, eg: -XX:MaxRAMPercentage=90
proposed improvements
-
echo
cmd.Stderr
andcmd.Stdout
to the job log. One option is to do this whenCI_DEBUG_TRACE
is set, the other option is to assume that folk frequently only look at the job log when jobs fail, and so this information is required for the most common use case. -
-quiet
is passed to spotbugs. The spotbugs documentation states that it's purpose is to:suppress error messages
. Remove this parameter and ensure error messages are surfaced in the job log.
The commit linked above includes most or all of these changes. (modified file attached: analyze.go)
This issue is raised as a bug, not feature request, because when this analyzer fails in this way, no information is provided for either the customer or GitLab support to fix it.
Other improvements to logging might also be beneficial, but that requires the pertinent output to be generated.
Steps to reproduce
(How one can reproduce the issue - this is very important)
Example Project
(If possible, please create an example project here on GitLab.com that exhibits the problematic behavior, and link to it here in the bug report)
(If you are using an older version of GitLab, this will also determine whether the bug is fixed in a more recent version)
What is the current bug behavior?
(What actually happens)
What is the expected correct behavior?
(What you should see instead)
Relevant logs and/or screenshots
(Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise.)
Output of checks
(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
(If you can, link to the line of code that might be responsible for the problem)