gemnasium-python fails to install psycopg2 (Dependency Scanning)

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

While trying to integrate dependency scanning in our (publicly visible) pipeline I get a failing build when gemnasium-python tries to installs our project requirements:

Error: pg_config executable not found.

The reason for this is that you need development prerequisites installed for the Postgres integration package psycopg2, which provides the pg_config executable (likewise for MySQL/MariaDB, by the way).

...
2020/02/17 14:58:38 [gemnasium-python] Starting analyzer...
Found project in /tmp/app
From https://gitlab.com/gitlab-org/security-products/gemnasium-db
 * branch              master     -> FETCH_HEAD
   b66ab3ea..99138979  master     -> origin/master
HEAD is now at 99138979 Merge branch 'adbcurate/CVE-2020-2124.yml' into 'master'
Collecting django-environ==0.4.5
  Downloading https://files.pythonhosted.org/packages/9f/32/76295a1a5d00bf556c495216581c6997e7fa5f533b2229e0a9d6cbaa95ae/django_environ-0.4.5-py2.py3-none-any.whl
  Saved ./dist/django_environ-0.4.5-py2.py3-none-any.whl
Collecting django-probes==1.2.0
  Downloading https://files.pythonhosted.org/packages/fa/c2/f41118c770f66dda3c388ecf63db28cf4e5b144197c4d79c4df3a90df958/django_probes-1.2.0.tar.gz
  Saved ./dist/django_probes-1.2.0.tar.gz
Collecting django==2.2.9
  Downloading https://files.pythonhosted.org/packages/cb/c9/ef1e25bdd092749dae74c95c2707dff892fde36e4053c4a2354b2303be10/Django-2.2.9-py3-none-any.whl (7.5MB)
  Saved ./dist/Django-2.2.9-py3-none-any.whl
Collecting pytz==2019.3
  Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB)
  Saved ./dist/pytz-2019.3-py2.py3-none-any.whl
Collecting sqlparse==0.3.0
  Downloading https://files.pythonhosted.org/packages/ef/53/900f7d2a54557c6a37886585a91336520e5539e3ae2423ff1102daf4f3a7/sqlparse-0.3.0-py2.py3-none-any.whl
  Saved ./dist/sqlparse-0.3.0-py2.py3-none-any.whl
Collecting uwsgi==2.0.18
  Downloading https://files.pythonhosted.org/packages/e7/1e/3dcca007f974fe4eb369bf1b8629d5e342bb3055e2001b2e5340aaefae7a/uwsgi-2.0.18.tar.gz (801kB)
  Saved ./dist/uwsgi-2.0.18.tar.gz
Collecting psycopg2==2.8.4
  Downloading https://files.pythonhosted.org/packages/84/d7/6a93c99b5ba4d4d22daa3928b983cec66df4536ca50b22ce5dcac65e4e71/psycopg2-2.8.4.tar.gz (377kB)
  Saved ./dist/psycopg2-2.8.4.tar.gz
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-download-d3azdy5y/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-download-d3azdy5y/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-download-d3azdy5y/psycopg2/pip-egg-info
         cwd: /tmp/pip-download-d3azdy5y/psycopg2/
    Complete output (23 lines):
    running egg_info
    creating /tmp/pip-download-d3azdy5y/psycopg2/pip-egg-info/psycopg2.egg-info
    writing /tmp/pip-download-d3azdy5y/psycopg2/pip-egg-info/psycopg2.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-download-d3azdy5y/psycopg2/pip-egg-info/psycopg2.egg-info/dependency_links.txt
    writing top-level names to /tmp/pip-download-d3azdy5y/psycopg2/pip-egg-info/psycopg2.egg-info/top_level.txt
    writing manifest file '/tmp/pip-download-d3azdy5y/psycopg2/pip-egg-info/psycopg2.egg-info/SOURCES.txt'
    
    Error: pg_config executable not found.
    
    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:
    
        python setup.py build_ext --pg-config /path/to/pg_config build ...
    
    or with the pg_config option in 'setup.cfg'.
    
    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.
    
    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
exit status 1
2020/02/17 14:58:47 Container exited with non zero status code

In other words, with the current setup of Dependency Scanning on GitLab, projects that require the psycopg2 Python package will be unable to run a successful dependency scan.

Is there a way to make this work that I didn't see?