SAST spotbugs setting SAST_JAVA_VERSION: 11 does not work
Summary
sast-spotbugs
is not setting java version correctly when switching to Java 11 using SAST_JAVA_VERSION
in config. See also #205696 (closed).
Steps to reproduce
- create a sast job in
.gitlab-ci.yml
by using the template as described in docs. - set
SAST_JAVA_VERSION
to11
in variables, doesn't matter if globally or on thesast
job itself. - set
SECURE_LOG_LEVEL
todebug
otherwise you will just seeexit 1 failed
, see also #216942 (closed). - try to build a java 11 project.
GitLab version is not that much relevant, as the issue seems to be in registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2
docker image. So all GitLab versions using the spotbugs:2
image might be affected.
Error:
...
Using Docker executor with image registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2 ...
Pulling docker image registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2 ...
Using docker image sha256:3988d5516c43c4bb9cf909b892a410d7f82811a2baa8e2472c94d25ee1873726 for registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2 ...
...
$ /analyzer run
[INFO] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ GitLab Find Security Bugs analyzer v2.13.2
[INFO] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ Detecting project
[INFO] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ Found project in /builds/am/sample-tracking
[INFO] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ Running analyzer
[DEBU] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ /bin/bash -c source /root/.bashrc && switch_to java adoptopenjdk-11.0.8+10
[INFO] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ Found Gradlew project in /builds/am/sample-tracking directory
[INFO] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ Found 1 analyzable projects.
[INFO] [Find Security Bugs] [2020-09-29T18:53:33Z] ▶ Building Gradlew project at /builds/am/sample-tracking.
[DEBU] [Find Security Bugs] [2020-09-29T18:54:01Z] ▶ /builds/am/sample-tracking/gradlew build
Welcome to Gradle 6.6.1!
Here are the highlights of this release:
- Experimental build configuration caching
- Built-in conventions for handling credentials
- Java compilation supports --release flag
For more details see https://docs.gradle.org/6.6.1/release-notes.html
To honour the JVM settings for this build a new JVM will be forked. Please consider using the daemon: https://docs.gradle.org/6.6.1/userguide/gradle_daemon.html.
Daemon will be stopped at the end of the build stopping after processing
locking FileBasedConfig[/root/.gitconfig] failed after 5 retries
[INFO] Some tasks have been disabled for job `spotbugs-sast`.
> Task :jrubyPrepare
Successfully installed asciidoctor-diagram-1.2.0-java
1 gem installed
> Task :generateLombokConfig
> Task :compileJava FAILED
> Task :processResources
> Task :processTestResources
> Task :jacocoTestReport SKIPPED
> Task :jacocoTestCoverageVerification SKIPPED
> Task :checkOutcomes SKIPPED
> Task :licenseMain UP-TO-DATE
> Task :licenseTest UP-TO-DATE
> Task :license UP-TO-DATE
> Task :spotlessGroovyGradle SKIPPED
> Task :spotlessGroovyGradleCheck SKIPPED
> Task :spotlessJava SKIPPED
> Task :spotlessJavaCheck SKIPPED
> Task :spotlessMisc SKIPPED
> Task :spotlessMiscCheck SKIPPED
> Task :spotlessCheck SKIPPED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':compileJava'.
> Could not target platform: 'Java SE 11' using tool chain: 'JDK 8 (1.8)'.
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 27s
7 actionable tasks: 5 executed, 2 up-to-date
[ERRO] [Find Security Bugs] [2020-09-29T18:54:01Z] ▶ Project couldn't be built: exit status 1
[FATA] [Find Security Bugs] [2020-09-29T18:54:01Z] ▶ exit status 1
Uploading artifacts for failed job
00:02
Uploading artifacts...
WARNING: gl-sast-report.json: no matching files
ERROR: No files to upload
ERROR: Job failed: exit code 1
What is the current bug behavior?
/analyzer
is not setting/using the requested java version.
What is the expected correct behavior?
/analyzer
is using the requested java version.
Relevant logs and/or screenshots
See above example in steps-to-reproduce.
Possible fixes
- Specify the java version with prefix as
adoptopenjdk-11
(only works if you manually invoke the image, as the/analyzer run
script does not allow it), see:
...
[INFO] [Find Security Bugs] [2020-09-29T19:50:20Z] ▶ GitLab Find Security Bugs analyzer v2.13.2
[INFO] [Find Security Bugs] [2020-09-29T19:50:20Z] ▶ Detecting project
[INFO] [Find Security Bugs] [2020-09-29T19:50:20Z] ▶ Found project in /builds/am/sample-tracking
[INFO] [Find Security Bugs] [2020-09-29T19:50:20Z] ▶ Running analyzer
[WARN] [Find Security Bugs] [2020-09-29T19:50:20Z] ▶ Java version adoptopenjdk-11 is not supported. Valid values are 8, 11. Using Java 8.
[DEBU] [Find Security Bugs] [2020-09-29T19:50:20Z] ▶ /bin/bash -c source /root/.bashrc && switch_to java adoptopenjdk-8.0.265+1
[INFO] [Find Security Bugs] [2020-09-29T19:50:21Z] ▶ Found Gradlew project in /builds/am/sample-tracking directory
[INFO] [Find Security Bugs] [2020-09-29T19:50:21Z] ▶ Found 1 analyzable projects.
...
- Remove the
^
in/root/.bashrc
switch_to
function and runversion="$(grep "$tool" "/root/.tool-versions"| tr ' ' '\n' | grep "$major_version")" # drop ^
You can test this locally by invoking the spotbugs image locally:
docker run \
-it --rm \
--env CI_PROJECT_DIR="/code" \
--env CI_JOB_NAME="local-spotbugs-sast" \
--env GRADLE_PATH="/code/gradlew" \
--env GRADLE_USER_HOME="/cache" \
--env SAST_JAVA_VERSION="11" \
--env SECURE_LOG_LEVEL="debug" \
--env SAST_EXCLUDED_PATHS="/code/.gradle,/code/build" \
--env JAVA_OPTS="-XX:MaxRAMPercentage=90" \
--volume $(pwd):/code:delegated \
--volume $HOME/.cache/gradle:/cache:delegated \
--entrypoint=bash \
registry.gitlab.com/gitlab-org/security-products/analyzers/spotbugs:2
Now you should have an instance of spotbugs container and bash ready to execute your commands
bash-5.0# java -version
openjdk version "1.8.0_265"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_265-b01)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.265-b01, mixed mode)
bash-5.0# env|grep -i java
JAVA_OPTS=-XX:MaxRAMPercentage=90
SAST_JAVA_VERSION=11
Switch to Java 11 by using the script from .bashrc
:
bash-5.0# cat /root/.bashrc
#!/bin/bash -l
update_java_home() {
local java_path
java_path="$(asdf which java)"
if [[ -n "${java_path}" ]]; then
export JAVA_HOME
JAVA_HOME="$(dirname "$(dirname "$(realpath "${java_path}")")")"
fi
}
function switch_to_exact() {
local tool=$1
local version=$2
asdf shell "$tool" "$version"
if [[ "$tool" = "java" ]]; then
update_java_home
fi
}
function switch_to() {
local tool=$1
local major_version=$2
local version
version="$(grep "$tool" "/root/.tool-versions"| tr ' ' '\n' | grep "^$major_version")" ## <=== this line causing the bug
switch_to_exact "$tool" "$version"
}
# shellcheck source=/dev/null
. "$ASDF_DATA_DIR/asdf.sh"
# shellcheck source=/dev/null
. "$ASDF_DATA_DIR"/plugins/java/set-java-home.bash
# shellcheck source=/dev/null
. "$ASDF_DATA_DIR"/plugins/ant/set-ant-home.bashbash-5.0
Please note, ^$major_version
will never match if you just specify the major version number, as the name intends.
bash-5.0# cat .tool-versions
# the first version listed for a given tool is the default
ant 1.10.8
java adoptopenjdk-8.0.265+1 adoptopenjdk-11.0.8+10
gradle 6.6.1
grails 4.0.4
maven 3.6.3
sbt 1.3.13
# Try with major version only
bash-5.0# switch_to java 11
bash-5.0# echo $?
0 # ==> so no error :-)
bash-5.0# env|grep -i java
JAVA_HOME=/opt/asdf/installs/java/adoptopenjdk-8.0.265+1 # ==> so no change :-(
JAVA_OPTS=-XX:MaxRAMPercentage=90
ASDF_JAVA_VERSION=
SAST_JAVA_VERSION=11
# Try including the prefix
bash-5.0# switch_to java adoptopenjdk-11
bash-5.0# env|grep -i java
JAVA_HOME=/opt/asdf/installs/java/adoptopenjdk-11.0.8+10 # ==> so changed :-)
JAVA_OPTS=-XX:MaxRAMPercentage=90
ASDF_JAVA_VERSION=adoptopenjdk-11
SAST_JAVA_VERSION=11