Dependency scanning fails for setuptools project that includes pyproject.toml
Summary
Python projects that use setuptools
for metadata (i.e. have a setup.cfg
or setup.py
file) but also include a pyproject.toml
file cannot be scanned properly by the dependency scanning template.
Note that including a pyproject.toml
file to declare the build-system
is the recommended way to configure a project for setuptools.
Steps to reproduce
Configure a project with the following files:
pyproject.toml
[build-system]
requires = [
"setuptools>=42",
"wheel",
]
build-backend = "setuptools.build_meta"
setup.py
from setuptools import setup
setup(
name="testproject",
version="0.0.1",
install_requires=["coloredlogs"],
)
.gitlab-ci.yml
include:
# https://docs.gitlab.com/ee/user/application_security/dependency_scanning/
- template: Jobs/Dependency-Scanning.gitlab-ci.yml
dependency_scanning:
variables:
SECURE_LOG_LEVEL: "debug"
Example Project
https://gitlab.com/duncanmmacleod/dependency-scanning-test/-/tree/setup.py
What is the current bug behavior?
gemnasium-python
see pyproject.toml
and presumes the project is a poetry project, and ignores the setup.py
file.
What is the expected correct behavior?
Since pyproject.toml
is independent of the build infrastructure it should be selected after setup.{cfg,py}
, probably last except for requirements.txt
.
Or, the scanner should read the pyproject.toml
file and use the [build-system]/build-backend
option to determine the package manager.
Relevant logs and/or screenshots
From https://gitlab.com/duncanmmacleod/dependency-scanning-test/-/jobs/2325799205:
$ /analyzer run
[INFO] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ GitLab gemnasium-python analyzer v2.22.0
[DEBU] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ inspect directory: .
[DEBU] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ skip ignored directory: .git
[DEBU] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ electing poetry for pypi because this is the first match
[DEBU] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ rejecting setup.py as handled by setuptools
[DEBU] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ electing poetry for pypi because this is the first match
[DEBU] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ rejecting setup.py as handled by setuptools
[INFO] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ Detected supported dependency files in '.'. Dependency files detected in this directory will be processed. Dependency files in other directories will be skipped.
[DEBU] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ Exporting dependencies for /builds/duncanmmacleod/dependency-scanning-test/pyproject.toml
[ERRO] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ No builder for package manager poetry
[FATA] [gemnasium-python] [2022-04-12T12:57:35Z] ▶ no builder for requirements file
Output of checks
This bug happens on GitLab.com
Possible fixes
pyproject.toml
is no longer specific to Poetry, so it should be removed from the definition of PackageManagerPoetry
.
// PackageManagerPoetry describes Poetry (Python)
PackageManagerPoetry = PackageManager{
Name: "poetry",
PackageType: PackageTypePypi,
Files: []File{
{Filename: "pyproject.toml", FileType: FileTypeRequirements},
{Filename: "poetry.lock", FileType: FileTypeLockFile},
},
}
Implementation plan
Update gemnasium-python:
- Remove
pyproject.toml
fromPackageManagerPoetry
. - Add an integration test for the case where a setuptool project has
setup.py
andpyproject.toml
. - Release a new version.