Dependency Scanning fails to build Python apps because of missing tools and dependencies
I am investigating the possibilities of dependency check and switching to Enterprise version of Gitlab.
I found that dependency check for python projects is based on analysis of requirements.txt file. If it contains common entries like 'Werkzeug==0.12.1', 'numpy==1.13.3' and so on, all is fine. But if there are next entries in requirements.txt:
cx-Oracle==5.3
psycopg2==2.6.2
pymssql==2.1.3the dependency check fails and exits. It is blocking factor for switching to enterprise version, because dependency check is broken by design.
docker run --interactive --tty --rm --volume "$PWD":/code --volume /var/run/docker.sock:/var/run/docker.sock registry.gitlab.com/gitlab-org/security-products/dependency-scanning:10-8-stable /code
EXECUTE: mkdir -p /app/bin
curl https://gitlab.com/gitlab-org/security-products/binaries/raw/master/gemnasium-client/gemnasium-client-1.0.1 --output /app/bin/gemnasium
chmod a+rx /app/bin/gemnasium
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 6949k 100 6949k 0 0 586k 0 0:00:11 0:00:11 --:--:-- 292k
EXECUTE: [ ! -z "$(/app/bin/gemnasium search .)" ]
EXECUTE: /app/bin/gemnasium alerts . > /code/gl-sast-gemnasium.json
latest: Pulling from gitlab-org/security-products/binaries/gemnasium-client-python-generator
Digest: sha256:c33ab2878c3776e291353e4078adcf6684d844e2bc25153ee5f02be22752446a
Status: Image is up to date for registry.gitlab.com/gitlab-org/security-products/binaries/gemnasium-client-python-generator:latest
-----> Installing python-3.6.4
-----> Installing pip
-----> Installing requirements with pip
Collecting airflow==1.8.0 (from -r /tmp/app/./requirements.txt (line 1))
Downloading https://files.pythonhosted.org/packages/e7/ac/5f1ec362fc0695167d29b3c7b6f28d79898f1221e5a32ab1c6e651a55564/airflow-1.8.0.tar.gz (8.4MB)
Collecting alembic==0.9.1 (from -r /tmp/app/./requirements.txt (line 2))
Downloading https://files.pythonhosted.org/packages/97/00/3e6797a2e4209db69d23b223ae3148d5f3605dafba6a332670de7a12c147/alembic-0.9.1.tar.gz (999kB)
Collecting amqp==2.1.4 (from -r /tmp/app/./requirements.txt (line 3))
Downloading https://files.pythonhosted.org/packages/7e/4b/ac7afb11b57f237e3c1c64b5408c5d229bf5d4b42af6cb6e683c7690ca4f/amqp-2.1.4-py2.py3-none-any.whl (49kB)
Collecting anyjson==0.3.3 (from -r /tmp/app/./requirements.txt (line 4))
Downloading https://files.pythonhosted.org/packages/c3/4d/d4089e1a3dd25b46bebdb55a992b0797cff657b4477bc32ce28038fdecbc/anyjson-0.3.3.tar.gz
Collecting appdirs==1.4.3 (from -r /tmp/app/./requirements.txt (line 5))
Downloading https://files.pythonhosted.org/packages/56/eb/810e700ed1349edde4cbdc1b2a21e28cdf115f9faf263f6bbf8447c1abf3/appdirs-1.4.3-py2.py3-none-any.whl
Collecting Babel==1.3 (from -r /tmp/app/./requirements.txt (line 6))
Downloading https://files.pythonhosted.org/packages/33/27/e3978243a03a76398c384c83f7ca879bc6e8f1511233a621fcada135606e/Babel-1.3.tar.gz (3.4MB)
Collecting bcrypt==3.1.2 (from -r /tmp/app/./requirements.txt (line 7))
Downloading https://files.pythonhosted.org/packages/3f/72/980f6e49da4ee3b168b20551e76142ad44af12318ed7e2d42ac0fd134b95/bcrypt-3.1.2-cp36-cp36m-manylinux1_x86_64.whl (53kB)
Collecting billiard==3.5.0.2 (from -r /tmp/app/./requirements.txt (line 8))
Downloading https://files.pythonhosted.org/packages/af/56/90fd158263e324742fb0ac82f9e2650dbbc7f93a233d9e254021e5d35880/billiard-3.5.0.2-py3-none-any.whl (102kB)
Collecting celery==3.1.23 (from -r /tmp/app/./requirements.txt (line 9))
Downloading https://files.pythonhosted.org/packages/de/df/59f5df67082ef46b86bc754b82f8cf187b835eea8a56ea8907813e75ad6d/celery-3.1.23-py2.py3-none-any.whl (520kB)
Collecting cffi==1.9.1 (from -r /tmp/app/./requirements.txt (line 10))
Downloading https://files.pythonhosted.org/packages/f0/47/2b967857a94b01127742dec3ed5595a596358cfbb170be6e3e89efd6786d/cffi-1.9.1-cp36-cp36m-manylinux1_x86_64.whl (398kB)
Collecting chartkick==0.4.2 (from -r /tmp/app/./requirements.txt (line 11))
Downloading https://files.pythonhosted.org/packages/2f/ce/b3d286e42fe5becc242e1c0e1f5a2365fa08546dd28155493571babf56fd/chartkick-0.4.2.tar.gz
Collecting click==6.7 (from -r /tmp/app/./requirements.txt (line 12))
Downloading https://files.pythonhosted.org/packages/34/c1/8806f99713ddb993c5366c362b2f908f18269f8d792aff1abfd700775a77/click-6.7-py2.py3-none-any.whl (71kB)
Collecting configparser==3.5.0 (from -r /tmp/app/./requirements.txt (line 13))
Downloading https://files.pythonhosted.org/packages/7c/69/c2ce7e91c89dc073eb1aa74c0621c3eefbffe8216b3f9af9d3885265c01c/configparser-3.5.0.tar.gz
Collecting croniter==0.3.16 (from -r /tmp/app/./requirements.txt (line 14))
Downloading https://files.pythonhosted.org/packages/58/2a/17d003f2a9a0188cf9365d63b3351c6522b7d83996b70270c65c789e35b9/croniter-0.3.16.tar.gz
Collecting cryptography==1.7.1 (from -r /tmp/app/./requirements.txt (line 15))
Downloading https://files.pythonhosted.org/packages/82/f7/d6dfd7595910a20a563a83a762bf79a253c4df71759c3b228accb3d7e5e4/cryptography-1.7.1.tar.gz (420kB)
Collecting cx-Oracle==5.3 (from -r /tmp/app/./requirements.txt (line 16))
Downloading https://files.pythonhosted.org/packages/14/05/4d492fb049eeee24ff8b5fdf23c6240b81ef168d4039dfbf6629e022ba6b/cx_Oracle-5.3.tar.gz (129kB)
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-wff2m3v5/cx-Oracle/setup.py", line 174, in <module>
raise DistutilsSetupError("cannot locate an Oracle software " \
distutils.errors.DistutilsSetupError: cannot locate an Oracle software installation
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-wff2m3v5/cx-Oracle/
Could not install python packages for the repository at .
FATA[0063] Container exited with non zero exit code: 1
/usr/local/lib/ruby/2.3.0/json/common.rb:156:in `initialize': A JSON text must at least contain two octets! (JSON::ParserError)
from /usr/local/lib/ruby/2.3.0/json/common.rb:156:in `new'
from /usr/local/lib/ruby/2.3.0/json/common.rb:156:in `parse'
from /app/lib/analyzers/gemnasium.rb:58:in `block in analyze'
from /app/lib/analyzers/gemnasium.rb:53:in `chdir'
from /app/lib/analyzers/gemnasium.rb:53:in `analyze'
from /app/lib/analyzers/gemnasium.rb:37:in `execute'
from /app/lib/analyze.rb:22:in `issues'
from /app/lib/run.rb:10:in `initialize'
from /app/bin/run:7:in `new'
from /app/bin/run:7:in `<main>'Edited by Fabio Busatto