dependency_scanning + sbt: shouldn't use `ivyReport` with sbt 1.3.0+
Summary
How much effort is it to support both sbt 1.3.0+ and the older version (as we do today) as there are changes to the way it worked? [timebox]
Since sbt 1.3.0 (Sep 4, 2019), Coursier became the default dependency management library, replacing Apache Ivy.
gemnasium-maven-dependency_scanning
happens to use ivyReport
to gather the project dependencies without considering the underlying dependency management library, which doesn't work with sbt 1.3.0+ unless Coursier
is explicitly disabled (ThisBuild / useCoursier := false
).
Steps to reproduce
Have the gemnasium-maven-dependency_scanning
job run on a sbt 1.3.0+ project (not disabling Coursier
!), for instance with the following project/build.properties
file:
sbt.version=1.3.13
Example Project
Any existing project may be quickly given a try by changing its sbt.version
as described above.
What is the current bug behavior?
sbt ivyReport
then sbt -no-colors show ivyReport
(misleadingly?) report successes, but post-processing their output fails because no report got actually generated:
[FATA] [gemnasium-maven] [<timestamp>] ▶ open /builds/<path>/target/scala-<scalaCompatVersion>/resolution-cache/reports/<groupId>.<artifactId>_<scalaCompatVersion>-compile.xml: no such file or directory
What is the expected correct behavior?
gemnasium-maven-dependency_scanning
doesn't make use of ivyReport
when detecting a sbt 1.3.0+ project (or by querying sbt
's active dependency management library).
Relevant logs and/or screenshots
[INFO] [gemnasium-maven] [<timestamp>] ▶ Identified sbt project
[DEBU] [gemnasium-maven] [<timestamp>] ▶ /opt/asdf/shims/sbt ivyReport
[info] [launcher] getting org.scala-sbt sbt 1.3.<patch> (this may take some time)...
:: loading settings :: url = jar:file:/opt/asdf/installs/sbt/1.3.8/bin/sbt-launch.jar!/org/apache/ivy/core/settings/ivysettings.xml
:: retrieving :: org.scala-sbt#boot-app
confs: [default]
<number> artifacts copied, 0 already retrieved
[info] Loading settings for project <artifactId> from <sbtFiles> ...
[info] Resolving key references (<number> settings) ...
[info] Set current project to <artifactId> (in build file:/builds/<path>/)
[success] Total time: <duration>, completed <dateTime>
[DEBU] [gemnasium-maven] [<timestamp>] ▶ /opt/asdf/shims/sbt -no-colors show ivyReport
[info] Loading settings for project <artifactId> from <sbtFiles> ...
[info] Resolving key references (<number> settings) ...
[info] Set current project to <artifactId> (in build file:/builds/<path>/)
[info] <artifactId> / Compile / ivyReport
[info] /builds/<path>/target/scala-<scalaCompatVersion>/resolution-cache/reports/<groupId>-<artifactId>_<scalaCompatVersion>-compile.xml
[success] Total time: <duration>, completed <dateTime>
[FATA] [gemnasium-maven] [<timestamp>] ▶ open /builds/<path>/target/scala-<scalaCompatVersion>/resolution-cache/reports/<groupId>.<artifactId>_<scalaCompatVersion>-compile.xml: no such file or directory
Uploading artifacts for failed job
00:00
Uploading artifacts...
WARNING: gl-dependency-scanning-report.json: no matching files
ERROR: No files to upload
ERROR: Job failed: exit code 1
Possible fixes
If possible, instead of using ivyReport
, gather dependencies in a generic way (independent from the underlying dependency manager), for instance with one of:
allDependencies
dependencyDot
dependencyList
dependencyTree
-
dependencyBrowseTree
(⚠ not headless) projectDependencies
If not, determine whether active dependency manager is Ivy
or Coursier
:
-
Ivy
: keep usingivyReport
-
Coursier
: either by means of one of the above dependency manager independent tasks, or by means of thesbt-coursier
plugin
EDIT: For sbt at least, the best alternative I'm aware of consists in leveraging the sbt-dependency-check
plugin which is based on the quite powerful OWASP dependency-check
, which in turn automatically updates itself using NVD Data Feeds.