If DS_PIP_VERSION is set gemnasium-python silently fails to install the selected version

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Work on this issue
• Close this issue

Summary

Steps to reproduce

1. Set up a project with python dependency scanning.
2. Trigger the scanning job with a DS_PIP_VERSION env var set.
3. Verify that a warning about setuptools is printed and pip is not installed.

Example Project

https://gitlab.com/madou-stories/pending-deletion/test-python-ds/

What is the current bug behavior?

The installPip function fails and the analyzer continues without any warning.

What is the expected correct behavior?

The get-pip.py script downloads and installs the correct version of pip specified. If it cannot install the version specified, the analyzer should fail loudly or warn loudly. The analyzer does not check the error value for the installPip function which hides the error. I think this surfaces another problem in our golangci-lint config - we don't enforce exhaustive error handling.

Relevant logs and/or screenshots

Error in job log

Click to expand

[DEBU] [gemnasium-python] [2023-10-19T14:20:53Z] [/go/src/app/builder/pip/pip.go:95] ▶ /usr/local/bin/python3 /get-pip.py --disable-pip-version-check --no-cache-dir pip==20.3.1
/usr/local/lib/python3.9/site-packages/_distutils_hack/__init__.py:18: UserWarning: Distutils was imported before Setuptools, but importing Setuptools also replaces the `distutils` module in `sys.modules`. This may lead to undesirable behaviors or errors. To avoid these issues, avoid using distutils directly, ensure that setuptools is installed in the traditional way (e.g. not an editable install), and/or make sure that setuptools is always imported before distutils.
  warnings.warn(
/usr/local/lib/python3.9/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")
Traceback (most recent call last):
  File "/get-pip.py", line 22317, in <module>
    main()
  File "/get-pip.py", line 198, in main
    bootstrap(tmpdir=tmpdir)
  File "/get-pip.py", line 120, in bootstrap
    import setuptools  # noqa
  File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 8, in <module>
    import _distutils_hack.override  # noqa: F401
  File "/usr/local/lib/python3.9/site-packages/_distutils_hack/override.py", line 1, in <module>
    __import__('_distutils_hack').do_override()
  File "/usr/local/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 77, in do_override
    ensure_local_distutils()
  File "/usr/local/lib/python3.9/site-packages/_distutils_hack/__init__.py", line 64, in ensure_local_distutils
    assert '_distutils' in core.__file__, core.__file__
AssertionError: /usr/local/lib/python3.9/distutils/core.py

Possible fixes

Replace the /get-pip.py script with a before_script that downloads the latest version of the script from https://bootstrap.pypa.io/get-pip.py.

rm -v /get-pip.py
curl -sSL -o /get-pip.py https://bootstrap.pypa.io/get-pip.py

This will ensure that the latest script/binary is installed and works. The drawback to this is that it does not work in offline mode without some configuration.

Implementation plan

• fix error handling https://gitlab.com/gitlab-org/security-products/analyzers/gemnasium/-/blob/cc62dbe8d2ab1829cf270f79e39e68a5eaf6bb2a/builder/pip/pip.go#L67
• update gemnasium-python with latest version of get-pip.py
  • get latest version from pypa
    • this will automatically rebuild the docker image with the correct version of the file
• document workaround using before_script to allow user to get their own version of get-pip.py

Testing

• Show previously failing job passing, eg:

  # 4.7.0 expected to fail
  
  # latest image expected to pass