Skip to content

Bug: Spotbugs Analyzer returns exit code 0 when analyzer fails to build

Summary

Our spotbugs SAST analyzer currently returns an exit code of 0 even when the project fails to build. When a build fails, an exit code of 1 should occur.

Steps to reproduce

Run analyzer against a test project with a broken pom; i.e. https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs/tree/master/test/broken and check $?

This can also be shown by manually executing the integration test ./test.sh

Example Project

https://gitlab.com/gitlab-org/security-products/analyzers/spotbugs/tree/master/test/broken

What is the current bug behavior?

Analyzer exits with 0 when failing to build project:

❯ IMAGE_TAG=analyzer CI_PROJECT_DIR=$PWD ./test.sh
1..3
No match in /tmp/project
ok 1 - Exit with exit status 3 when project not found
Found project in /tmp/project
Found Maven project in /tmp/project directory
Found 1 analyzable projects.
Building Maven project at /tmp/project.
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-parseable POM /tmp/project/pom.xml: only whitespace content allowed before start tag and not B (position: START_DOCUMENT seen B... @1:1)  @ line 1, column 1
 @
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project  (/tmp/project/pom.xml) has 1 error
    1 #!/bin/sh
[ERROR]     Non-parseable POM /tmp/project/pom.xml: only whitespace content allowed before start tag and not B (position: START_DOCUMENT seen B... @1:1)  @ line 1, column 1 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/ModelParseException
Error: Project couldn't be built: exit status 1

Warning: Building failed for /tmp/project. Attempting scan anyway.
No classfiles specified; output will have no warnings
Warning: SpotBugs didn't find any class file to analyze in /tmp/project !

not ok 2 - Exit with exit status 1 when there's something wrong
Failed 1/2 tests

What is the expected correct behavior?

❯ docker run   --interactive --tty --rm   --volume "$PWD":/tmp/app   --env CI_PROJECT_DIR=/tmp/app analyzer /analyzer run
Found 1 analyzable projects.
Building Maven project at /tmp/project.
[INFO] Scanning for projects...
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-parseable POM /tmp/project/pom.xml: only whitespace content allowed before start tag and not B (position: START_DOCUMENT seen B... @1:1)  @ line 1, column 1
 @
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project  (/tmp/project/pom.xml) has 1 error
    1 #!/bin/sh
[ERROR]     Non-parseable POM /tmp/project/pom.xml: only whitespace content allowed before start tag and not B (position: START_DOCUMENT seen B... @1:1)  @ line 1, column 1 -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/ModelParseException
Error: Project couldn't be built: exit status 1

~/code/gl/security_products/tests/c master*
❯ echo $?
1

Relevant logs and/or screenshots

https://gitlab.com/gitlab-org/security-products/tests/java-maven/-/jobs/181319224

Possible fixes

The analyzer should fail if compilation fails (with the special case for Groovy, failing if both compilation fails).

Development log

Status

Decisions

Edited by Lucas Charles