Dependency Scanning of Pipfile.lock without installing project dependencies
Problem to solve
Dependency Scanning for Python (gemnasium-python) can scan pipenv projects and report vulnerabilities for the exact versions listed in Pipfile.lock
. However, it requires to install the project dependencies, and this consumes resources (time and bandwidth), and might fail if the environment of the scanning job doesn't meet all the requirements of the installed packages (version of python and system libraries).
Proposal
Parse Pipfile.lock
directly, without running pipenv graph
to export the dependency graph.
As a side effect, it becomes possible to scan a Pipenv project that doesn't have a Pipfile
.
Warning! Pipfile.lock
only contains normalized package names, like django
, and not canonical names, like Django
. Parsing the lock file might result in inaccurate package names. That said, vulnerabilities are accurately reported, thanks to PEP 426 support implemented in #33341 (closed).
Further details
gemnasium-python
runs pipenv graph --json
to export the dependency graph as a JSON document. This JSON doc is similar to the one pipdeptree
creates. pipenv graph
installs the project dependencies, and this might fail if the installed packages require a specific version of Python, or specific system libraries.
What does success look like, and how can we measure that?
- Less failing Dependency Scanning jobs for Python projects, more Python projects being scanned.
- Less time spent scanning Pipenv project
Links / references
Implementation plan
-
Add Pipfile.lock
parser to gemnasium #33227 (closed) -
Enable Pipfile.lock
parser ingemnasium-python
, and make sure thepipenv
builder is skipped when a lock file is provided -
Add exists
line to detection rule forPipfile.lock
in the Dependency Scanning template -
Update user docs to mention support
Testing
This is testing using test branch https://gitlab.com/gitlab-org/security-products/tests/python-pipenv/-/commits/pipfile-lock-FREEZE and the corresponding QA job.