Report vulnerable dependency paths for pip, setuptools (Python)

Problem to solve

Dependency Scanning should report the dependency paths for vulnerable dependencies found in Python projects using pip and setuptools (no lock file). These dependency paths can then be shown in the UI, including in the dependency list. See #227620 (closed)

Proposal

Update the lock file parser used to parse the JSON output of pipdeptree, and make it able to build the dependency graph.

Proposal 2

requirements.txt generated by the pip-compile acts as a kind of lock file to dependency configuration stored pyproject.toml, setuptools.py, etc. Because requirements.txt would be committed to the repo like a regular lockfile we can support user projects without asking them to add extra processing steps, like generating an artifact or using an extra tool like `pipdeptree`.

It also provides a flexible solution for other build tools without an official lockfile (`setuptools`) and is actually quite flexible.

For this reason proposal 2 is chosen over the original.

Implementation plan

Sample requirements.txt generated by pip-compile:

#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
#    pip-compile
#
asgiref==3.8.1
    # via django
django==5.1.1
    # via -r requirements.in
sqlparse==0.5.1
    # via django
  • Add parser for requirements.txt files generated with the pip-compile header.
  • Find packages by finding lines matching <name>==<version> format.
  • Find dependencies by finding lines matching # via <name>.
  • Add integration test to verify the generated sbom.

Documentation

Document that the requirements.txt that is expected is from the pip-compile tool.

Edited by Igor Frenkel