Skip to content

Dependency Scanning of Pipfile.lock without installing project dependencies

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Problem to solve

Dependency Scanning for Python (gemnasium-python) can scan pipenv projects and report vulnerabilities for the exact versions listed in Pipfile.lock. However, it requires to install the project dependencies, and this consumes resources (time and bandwidth), and might fail if the environment of the scanning job doesn't meet all the requirements of the installed packages (version of python and system libraries).

Proposal

Parse Pipfile.lock directly, without running pipenv graph to export the dependency graph.

As a side effect, it becomes possible to scan a Pipenv project that doesn't have a Pipfile.

Warning! Pipfile.lock only contains normalized package names, like django, and not canonical names, like Django. Parsing the lock file might result in inaccurate package names. That said, vulnerabilities are accurately reported, thanks to PEP 426 support implemented in #33341 (closed).

Further details

gemnasium-python runs pipenv graph --json to export the dependency graph as a JSON document. This JSON doc is similar to the one pipdeptree creates. pipenv graph installs the project dependencies, and this might fail if the installed packages require a specific version of Python, or specific system libraries.

What does success look like, and how can we measure that?

  • Less failing Dependency Scanning jobs for Python projects, more Python projects being scanned.
  • Less time spent scanning Pipenv project

Links / references

Implementation plan

  • Add Pipfile.lock parser to gemnasium #33227 (closed)
  • Enable Pipfile.lock parser in gemnasium-python, and make sure the pipenv builder is skipped when a lock file is provided
  • Add exists line to detection rule for Pipfile.lock in the Dependency Scanning template
  • Update user docs to mention support

Testing

This is testing using test branch https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/commits/pipfile-lock-FREEZE and the corresponding QA job.

Product Management - @NicoleSchwartz

cc @NicoleSchwartz @gonzoyumo @plafoucriere

Edited by 🤖 GitLab Bot 🤖