Skip to content

Poetry's pyproject.toml support for Dependency scanning

Problem to solve

Add Python Poetry support to Dependency Scanning, handle Poetry projects having a pyproject.toml but no poetry.lock lock file.

See #7006 (closed) for supporting poetry.lock.

Further details

Some Poetry Python don't have a lock file and thus can't be scanned by Dependency Scanning.

Proposal

Update analyzers/gemnasium-python so that it detects pyproject.toml, generates a poetry.lock using Poetry, and parses it using the dedicated dependency file parser. See #7006 (closed)

Warning! pyproject.toml must be ignored when poetry.lock is parsed! The lock file dynamically generated from pyproject.toml may differ from the poetry.lock checked in the repo, but the latter is more accurate for it reflects what the application will ultimately install and execute. Also, parsing both pyproject.toml and poetry.lock (respectively in gemnasium-python and gemnasium) would result in redundant, conflicting information.

It would be easier to coordinate pyproject.toml support with poetry.lock support if both were implemented in analyzers/gemnasium, but the gemnasium image may not be suited for Python projects. To be investigated. See also #13477 (closed)

Implementation Plan

Please note that this assumes the following tasks were accomplished in the poetry.lock support issue:

  • add poetry.lock parser in gemnasium
  • add test project

Backend

  • add poetry to gemnasium-python docker image
  • add new package manager type to gemnasium-python so that it can issue install command correctly
  • skip the poetry lock command if an existing poetry.lock file is colocated with the pyproject.toml file, and it specifies the build-backend as poetry.mason.api or poetry.core.masonry.api1. This ensures that we only build the lock file if absolutely needed.
  • add integration tests for poetry projects without a lock file
  • add **/pyproject.toml to the shared rules for gemnasium-python

What does success look like, and how can we measure that?

Dependency Scanning CI jobs successfully scans Poetry projects having no lock file instead of failing with "unsupported language or manager" error.

Links / references

  1. Poetry Documentation Archive ↩

Edited by Oscar Tovar