Skip to content

dependency_scanning + sbt: shouldn't use `ivyReport` with sbt 1.3.0+

Summary

How much effort is it to support both sbt 1.3.0+ and the older version (as we do today) as there are changes to the way it worked? [timebox]

Since sbt 1.3.0 (Sep 4, 2019), Coursier became the default dependency management library, replacing Apache Ivy.

gemnasium-maven-dependency_scanning happens to use ivyReport to gather the project dependencies without considering the underlying dependency management library, which doesn't work with sbt 1.3.0+ unless Coursier is explicitly disabled (ThisBuild / useCoursier := false).

Steps to reproduce

Have the gemnasium-maven-dependency_scanning job run on a sbt 1.3.0+ project (not disabling Coursier!), for instance with the following project/build.properties file:

sbt.version=1.3.13

Example Project

Any existing project may be quickly given a try by changing its sbt.version as described above.

What is the current bug behavior?

sbt ivyReport then sbt -no-colors show ivyReport (misleadingly?) report successes, but post-processing their output fails because no report got actually generated:

[FATA] [gemnasium-maven] [<timestamp>] ▶ open /builds/<path>/target/scala-<scalaCompatVersion>/resolution-cache/reports/<groupId>.<artifactId>_<scalaCompatVersion>-compile.xml: no such file or directory

What is the expected correct behavior?

gemnasium-maven-dependency_scanning doesn't make use of ivyReport when detecting a sbt 1.3.0+ project (or by querying sbt's active dependency management library).

Relevant logs and/or screenshots

[INFO] [gemnasium-maven] [<timestamp>] ▶ Identified sbt project
[DEBU] [gemnasium-maven] [<timestamp>] ▶ /opt/asdf/shims/sbt ivyReport
[info] [launcher] getting org.scala-sbt sbt 1.3.<patch>  (this may take some time)...
:: loading settings :: url = jar:file:/opt/asdf/installs/sbt/1.3.8/bin/sbt-launch.jar!/org/apache/ivy/core/settings/ivysettings.xml
:: retrieving :: org.scala-sbt#boot-app
	confs: [default]
	<number> artifacts copied, 0 already retrieved
[info] Loading settings for project <artifactId> from <sbtFiles> ...
[info] Resolving key references (<number> settings) ...
[info] Set current project to <artifactId> (in build file:/builds/<path>/)
[success] Total time: <duration>, completed <dateTime>
[DEBU] [gemnasium-maven] [<timestamp>] ▶ /opt/asdf/shims/sbt -no-colors show ivyReport
[info] Loading settings for project <artifactId> from <sbtFiles> ...
[info] Resolving key references (<number> settings) ...
[info] Set current project to <artifactId> (in build file:/builds/<path>/)
[info] <artifactId> / Compile / ivyReport
[info] 	/builds/<path>/target/scala-<scalaCompatVersion>/resolution-cache/reports/<groupId>-<artifactId>_<scalaCompatVersion>-compile.xml
[success] Total time: <duration>, completed <dateTime>
[FATA] [gemnasium-maven] [<timestamp>] ▶ open /builds/<path>/target/scala-<scalaCompatVersion>/resolution-cache/reports/<groupId>.<artifactId>_<scalaCompatVersion>-compile.xml: no such file or directory
Uploading artifacts for failed job
00:00
Uploading artifacts...
WARNING: gl-dependency-scanning-report.json: no matching files 
ERROR: No files to upload                          
ERROR: Job failed: exit code 1

Possible fixes

If possible, instead of using ivyReport, gather dependencies in a generic way (independent from the underlying dependency manager), for instance with one of:

  1. allDependencies
  2. dependencyDot
  3. dependencyList
  4. dependencyTree
  5. dependencyBrowseTree (⚠ not headless)
  6. projectDependencies

If not, determine whether active dependency manager is Ivy or Coursier:

  1. Ivy: keep using ivyReport
  2. Coursier: either by means of one of the above dependency manager independent tasks, or by means of the sbt-coursier plugin

EDIT: For sbt at least, the best alternative I'm aware of consists in leveraging the sbt-dependency-check plugin which is based on the quite powerful OWASP dependency-check, which in turn automatically updates itself using NVD Data Feeds.

Edited by Nicole Schwartz