Add dependency resolution job for python to DS CI template v2

What does this MR do and why?

Add a dependency Resolution job for python to the Dependency Scanning CI template v2 that attempts to automatically generate a pipcompile.lock.txt file for a project using various python requirements files: requirements.txt, requirements.pip, requires.txt, setup.py, setup.cfg, pyproject.toml

Python projects may not have a lockfile commited to their repository. Accurate dependency scanning requires one, which today users must generate themselves via a custom CI job. This creates friction and makes it impossible to enforce scanning policies at scale.

The solution relies on coordinating a main dependency resolution job container with the DS analyzer running as a service container:

  1. The resolution job checks out the repository and signals readiness by creating a .gitlab_ds/ready_${CI_JOB_ID} file
  2. The service container detects projects needing resolution and writes a shell script to .gitlab_ds/dependency_resolution.sh
  3. The resolution job executes the script, which runs the appropriate resolution command per project (e.g. uv pip compile for Python), producing graph files
  4. Those files are exported as artifacts and picked up by the main dependency scanning job that runs in a later stage.

This MR replicates the dependency resolution mechanism implementated for Maven in Add dependency resolution job for maven to DS C... (!226990 - merged)

References

Screenshots or screen recordings

Before After

How to set up and validate locally

  1. checkout this branch in your GDK
  2. clone this project: https://gitlab.com/gitlab-org/security-products/tests/python-pip
  3. adjust CI configuration to just use v2 template:
     include:
       - template: Jobs/Dependency-Scanning.v2.gitlab-ci.yml
  4. Run a pipeline and confirms that:
    1. a dependency-scanning:python-resolution job ran in .pre stage, emitting a pipcompile.lock.txt file
    2. the dependency-scanning job ran and successfully scanned this lock file.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Olivier Gonzalez

Merge request reports

Loading
Loading