Skip to content
GitLab
Next
    • GitLab: the DevOps platform
    • Explore GitLab
    • Install GitLab
    • How GitLab compares
    • Get started
    • GitLab docs
    • GitLab Learn
  • Pricing
  • Talk to an expert
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
    Projects Groups Snippets
  • Sign up now
  • Login
  • Sign in / Register
  • GitLab GitLab
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 46,778
    • Issues 46,778
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 1,533
    • Merge requests 1,533
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Artifacts
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.orgGitLab.org
  • GitLabGitLab
  • Issues
  • #7006
Closed
Open
Issue created Jul 24, 2018 by golyalpha@golyalpha12 of 12 checklist items completed12/12 checklist items

poetry.lock support for Dependency Scanning

Problem to solve

Add Python Poetry support to Dependency Scanning, parse and process Poetry lock files.

See #32774 for supporting Poetry projects having no lock file.

Right now gemnasium-python fails because it detects the Poetry project, but then tries to install its dependencies instead of parsing poetry.lock, and it can't find a compatible builder.

[INFO] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ GitLab gemnasium-python analyzer v2.23.0
[INFO] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ Detected supported dependency files in '.'. Dependency files detected in this directory will be processed. Dependency files in other directories will be skipped.
[ERRO] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ No builder for package manager poetry
[FATA] [gemnasium-python] [2022-04-14T13:19:01Z] ▶ no builder for requirements file

Further details

Poetry is a package manager and builder for Python, which is arguably better than pure pip. It generates poetry.lock, a lock file that gives the full, accurate list of the package versions a Python project depends on. Dependency Scanning should leverage this lock file to detect and report vulnerable dependencies.

Proposal

  • add poetry.lock parser to gemnasium; see gitlab-org/security-products/analyzers/gemnasium!66 (merged)
  • create python-poetry test project
  • add Poetry as a package manager detected by gemnasium's finder, and enable it in PresetGemnasiumPython.
  • (see implementation plan) integrate the poetry.lock parser into gemnasium-python, and leverage the python-poetry project for QA
  • (see implementation plan) update Dependency Scanning documentation

Implementation plan

  • update gemnasium-python's main.go to skip the "build" step when there's no builder, but a lock file parser is available. Right now it fails in that case.
  • pull poetry.lock parser into gemnasium-python similar to pipdeptree: https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/master/main.go#L30
  • update image spec https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium-python/-/blob/master/spec/image_spec.rb to add test for poetry (can use lock file from https://gitlab.com/gitlab-org/security-products/tests/python-poetry/-/blob/master/poetry.lock)
  • update CI template to trigger gemnasium-python-dependency_scanning when there's a poetry.lock, and test this in dependency_scanning_gitlab_ci_yaml_spec.rb
  • update documentation
    • document support for poetry: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#supported-languages-and-package-managers
    • document version tested: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#obtaining-dependency-information-by-parsing-lockfiles
    • remove additional support section: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#support-for-additional-languages
    • remove workaround for poetry: https://docs.gitlab.com/ee/user/application_security/dependency_scanning/#working-around-missing-support-for-certain-languages-or-package-managers

Availability & testing

python-poetry test project should be updated to test poetry.lock, test should pass.

What does success look like, and how can we measure that?

Dependency Scanning CI job successfully process Poetry projects having a lock file instead of failing with "unsupported language or manager" error.

Links / references

  • Poetry
Edited May 18, 2022 by Oscar Tovar
Assignee
Assign to
Time tracking