Enable development dependency scanning in pipenv projects

Release notes

TODO

Problem

As part of Add ability to optionally ignore dev dependenci... (#364585 - closed), the parser for Pipfile.lock files was updated to handle parsing the dev dependencies. By default, this would include new dependencies, and as such is considered a breaking change that must be released in %16.0. To accommodate this, the Pipfile.lock parser has not been added to gemnasium-python and must be added when releasing %16.0.

Proposal

Add the piplock parser to gemnasium-python and have it start scanning development dependencies when configured to do so.

NOTE: This would solve Dependency Scanning of Pipfile.lock without ins... (#299294).

Implementation

• Update Gemnasium.
  • Add the pipfilelock parser to cmd/gemansium-python/main.go.
  • Update the pipenv builder so that it does not export a pipdeptree.json file and instead only generates the Pipfile.lock file required for the parser.
  • Update integration tests to include a test case for DS_INCLUDE_DEV_DEPENDENCIES="false".
    • It should cover cases where there's a Pipfile.lock.
    • It should cover cases where there's only Pipfile.
    • It should cover cases where there's only a Pipfile.lock.
  • Release a new version of Gemnasium v4.
• Update the CI/CD templates to trigger gemnasium-python when a Pipfile.lock file is found at any level. Ideally this is change when changing DS_MAJOR_VERSION to 4, or after doing it. See #375505 (comment 1373691028)
• Update documentation Running dependency scanning in an offline environment so that it includes the steps necessary for offline deployments.
  • Using a PyPi Mirror documentation.
• Update documentation Configuring specific analyzers used by dependency scanning to mention support for Pipenv projects.

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.