Use pip installation report in Dependency Scanning

Problem to solve

Right now gemnasium-python relies on pipdeptree to list dependencies of Python projects after running pip or an install script based on setuptools.

However,

  • It's a dependency that needs to be maintained/updated.
  • Right now this makes the scan fails if the project being scanned depends on a different version of pipdeptree. In particular, a self-scan of gemnasium-python fails in an MR that updates pipdeptree, because the scan uses an image built from the master branch. See gitlab-org/security-products/analyzers/gemnasium!623 (comment 1743270995)
  • Also, installing pipdeptree is one more instructions for users who want to produce input files for Dependency Scanning.

Proposal

Make gemnasium-python generate and parse a pip installation report instead of running pipdeptree.

Further details

See #418321 (comment 1746439090)

The installation report format fully documented and provides the information we need: package name, version, dependencies/requirements.

Comparison with pipdeptree:

  • It doesn't directly give the requirement of the package itself, but this is fine. It's like Gemfile.lock in that way.
  • It gives additional information: extra packages, required version of Python, package URL, and wether this is a direct dependency. That's valuable information.

To be checked:

  • We have to make sure we can install versions of pip that have this "installation report" feature in all supported version of Python. It should be the case.
  • It might not provide canonical package names, like Django.

/cc @hacks4oats