Detect and report pre-installed versions of package managers in gemnasium

Problem to solve

As discussed in Update stale links in DS docs, in some cases, the pre-installed versions of package managers are not pinned by a variable or file, so they constantly change, and are impossible to document accurately.

For example, we currently document that the pre-installed version of sbt is 1.6.1 because this is the version that's pinned in the .tool-versions file, however, the versions of pip and setuptools are not so clearly defined, because they come from the python:3.9-slim base image used in gemnasium-python, and whenever this base image is updated and a new analyzer is released, the versions of pip and setuptools will change.

We can determine the current versions of these packages by inspecting the output of the gemnasium-python analyzer:

docker run -it --rm registry.gitlab.com/security-products/gemnasium-python pip list | grep 'setuptools\|pipenv'

pipenv              2022.1.8
setuptools          58.1.0

The purpose of this issue is to update the gemnasium* analyzers to output the versions of the pre-installed package managers, which will make it easier to document these values.

Proposal

  1. Determine which package manager versions to report.
  2. Determine the format for reporting the package manager versions. We could possibly report them to stdout in a human-readable format, while also saving them to a JSON file to allow these values to be programmatically ingested.
  3. Implement the necessary changes in gemnasium*.

Implementation Plan

Use this MR as a starting point.

  1. Create new toolversion package in gemnasium

  2. toolversion will be responsible for executing the commands necessary for obtaining the package manager versions and parsing the output:

    1. gemnasium

      • Package Manager: go
        • Command: go version
        • Output: go version go1.18.7 linux/amd64

    2. gemnasium-python

      • Package Manager(s): pip, pipenv, setuptools

        • Command: pip list --format=json

        • Output:

          [
  { "version": "22.0.4", "name": "pip" },
  { "version": "2.2.1", "name": "pipdeptree" },
  { "version": "2022.1.8", "name": "pipenv" },
  { "version": "58.1.0", "name": "setuptools" }
]

    3. gemnasium-maven

      • Package Manager: sbt

        • Command: sbt --numeric-version
        • Output: 1.6.1

      • Package Manager: gradle

        • Command: gradle --version

        • Output:

          ------------------------------------------------------------
Gradle 7.3.3
------------------------------------------------------------

Build time:   2021-12-22 12:37:54 UTC
Revision:     6f556c80f945dc54b50e0be633da6c62dbe8dc71

Kotlin:       1.5.31
Groovy:       3.0.9
Ant:          Apache Ant(TM) version 1.10.11 compiled on July 10 2021
JVM:          17.0.2 (Eclipse Adoptium 17.0.2+8)
OS:           Linux 5.15.49-linuxkit amd64

      • Package Manager: mvn

        • Command: mvn --version

        • Output:

          Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /opt/asdf/installs/maven/3.6.3
Java version: 17.0.2, vendor: Eclipse Adoptium, runtime: /opt/asdf/installs/java/adoptopenjdk-17.0.2+8
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "5.15.49-linuxkit", arch: "amd64", family: "unix"

  3. Add unit tests for toolversion

  4. Output package version information as an info level log message when running the analyzer. Note: for sbt, this will add 6 seconds to the analyzer execution time.

    The version we output should match the package manager being executed. For example, when analyzing a gomodules project, we should output the version of go, and when analyzing a python-setuptools project, we should output the version of setuptools.

  5. Add a new commandline flag to output package version information to a JSON formatted file.

    The following package manager version information should be output to this JSON file when executing the run command of each analyzer:

    • gemnasium
      • go
    • gemnasium-python
      • setuptools
      • pip
      • pipenv
    • gemnasium-maven
      • sbt
      • gradle
      • maven

    The contents of this JSON file will eventually be used to automatically generate Dependency Scanning documentation, so it needs to include multiple version values for each analyzer.

  6. Add image integration tests for steps 4. and 5. above.

/cc @fcatteau @gonzoyumo

Edited by Adam Cohen