poetry.lock support for Dependency Scanning
Problem to solve
Add Python Poetry support to Dependency Scanning, parse and process Poetry lock files.
See #32774 for supporting Poetry projects having no lock file.
gemnasium-python fails because it detects the Poetry project, but then tries to install its dependencies instead of parsing
poetry.lock, and it can't find a compatible
[INFO] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ GitLab gemnasium-python analyzer v2.23.0 [INFO] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ Detected supported dependency files in '.'. Dependency files detected in this directory will be processed. Dependency files in other directories will be skipped. [ERRO] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ No builder for package manager poetry [FATA] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ no builder for requirements file
Poetry is a package manager and builder for Python, which is arguably better than pure pip. It generates
poetry.lock, a lock file that gives the full, accurate list of the package versions a Python project depends on. Dependency Scanning should leverage this lock file to detect and report vulnerable dependencies.
poetry.lockparser to gemnasium; see gitlab-org/security-products/analyzers/gemnasium!66 (merged)
create python-poetry test project
add Poetry as a package manager detected by
gemnasium's finder, and enable it in
- (see implementation plan) integrate the
poetry.lockparser into gemnasium-python, and leverage the python-poetry project for QA
- (see implementation plan) update Dependency Scanning documentation
main.goto skip the "build" step when there's no
builder, but a lock file parser is available. Right now it fails in that case.
update image spec https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/master/spec/image_spec.rb to add test for
poetry(can use lock file from https://gitlab.com/gitlab-org/security-products/tests/python-poetry/-/blob/master/poetry.lock)
update CI template to trigger
gemnasium-python-dependency_scanningwhen there's a
poetry.lock, and test this in dependency_scanning_gitlab_ci_yaml_spec.rb
document support for
document version tested: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#obtaining-dependency-information-by-parsing-lockfiles
remove additional support section: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#support-for-additional-languages
remove workaround for
Availability & testing
python-poetry test project should be updated to test poetry.lock, test should pass.
What does success look like, and how can we measure that?
Dependency Scanning CI job successfully process Poetry projects having a lock file instead of failing with "unsupported language or manager" error.