PyPi package registry provides problematic pip command line
Summary
Installation of python packages that have dependencies not hosted in the GitLab PyPi package registry fail to install using provided installation command.
The details page for a python package in the GitLab PyPi package registry provides a problematic installation command. The command provided uses the --index-url
parameter to include the GitLab PyPi package registry. This prevents pip
from using any other package registry. This prevents dependencies from installing if they are not also hosted in the GitLab PyPi package registry.
Example output showing the install failing:
python -m pip install gitlab-apifuzzing-recorder --index-url https://gitlab.com/api/v4/projects/19617580/packages/pypi/simple
Looking in indexes: https://gitlab.com/api/v4/projects/19617580/packages/pypi/simple
Collecting gitlab-apifuzzing-recorder
Downloading https://gitlab.com/api/v4/projects/19617580/packages/pypi/files/6136450aa662012d84d35b88c5bec36fc9654d1dadd89ec14340968642975d13/gitlab_apifuzzing_recorder-1.6.33-py3-none-any.whl (8.6 kB)
ERROR: Could not find a version that satisfies the requirement mitmproxy~=4.0.4 (from gitlab-apifuzzing-recorder) (from versions: none)
ERROR: No matching distribution found for mitmproxy~=4.0.4 (from gitlab-apifuzzing-recorder)
Steps to reproduce
- Upload a python package that has dependencies on packages hosted in the public PyPi registry
- Go to the packages detail page in GitLab
- Run the Installation Pip Command. Notice the command uses
--index-url
. - Receive error install package because the dependencies are not found
Example Project
Here is a project and package showing this behaviour:
https://gitlab.com/gitlab-org/security-products/analyzers/api-fuzzing/-/packages/505181
What is the current bug behavior?
Packages with dependencies from the public PyPi registry will not install.
What is the expected correct behavior?
Packages with dependencies from the public PyPi registry install without error.
Relevant logs and/or screenshots
Provided installation command:
Example output showing the install failing:
python -m pip install gitlab-apifuzzing-recorder --index-url https://gitlab.com/api/v4/projects/19617580/packages/pypi/simple
Looking in indexes: https://gitlab.com/api/v4/projects/19617580/packages/pypi/simple
Collecting gitlab-apifuzzing-recorder
Downloading https://gitlab.com/api/v4/projects/19617580/packages/pypi/files/6136450aa662012d84d35b88c5bec36fc9654d1dadd89ec14340968642975d13/gitlab_apifuzzing_recorder-1.6.33-py3-none-any.whl (8.6 kB)
ERROR: Could not find a version that satisfies the requirement mitmproxy~=4.0.4 (from gitlab-apifuzzing-recorder) (from versions: none)
ERROR: No matching distribution found for mitmproxy~=4.0.4 (from gitlab-apifuzzing-recorder)
Possible fixes
Instead of --index-url
use --extra-index-url
on provided pip command line.
Example of fix working with example project/package:
$ python -m pip install gitlab-apifuzzing-recorder --extra-index-url https://gitlab.com/api/v4/projects/19617580/packages/pypi/simple
Looking in indexes: https://pypi.org/simple, https://gitlab.com/api/v4/projects/19617580/packages/pypi/simple
Collecting gitlab-apifuzzing-recorder
Using cached https://gitlab.com/api/v4/projects/19617580/packages/pypi/files/6136450aa662012d84d35b88c5bec36fc9654d1dadd89ec14340968642975d13/gitlab_apifuzzing_recorder-1.6.33-py3-none-any.whl (8.6 kB)
Collecting click~=6.7
Using cached click-6.7-py2.py3-none-any.whl (71 kB)
Collecting mitmproxy~=4.0.4
Downloading mitmproxy-4.0.4-py3-none-any.whl (1.3 MB)
|████████████████████████████████| 1.3 MB 7.4 MB/s
Collecting pyOpenSSL<18.1,>=17.5
Downloading pyOpenSSL-18.0.0-py2.py3-none-any.whl (53 kB)
|████████████████████████████████| 53 kB 1.5 MB/s
Collecting ldap3<2.6,>=2.5
Downloading ldap3-2.5.2-py2.py3-none-any.whl (393 kB)
|████████████████████████████████| 393 kB 32.2 MB/s
Collecting urwid<2.1,>=2.0.1
Downloading urwid-2.0.1.tar.gz (604 kB)
|████████████████████████████████| 604 kB 44.7 MB/s
Collecting pyperclip<1.7,>=1.6.0
Downloading pyperclip-1.6.5.tar.gz (15 kB)
Collecting certifi>=2015.11.20.1
Using cached certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Collecting cryptography<2.4,>=2.1.4
Downloading cryptography-2.3.1-cp34-abi3-manylinux1_x86_64.whl (2.1 MB)
|████████████████████████████████| 2.1 MB 25.2 MB/s
Collecting hyperframe<6,>=5.1.0
Downloading hyperframe-5.2.0-py2.py3-none-any.whl (12 kB)
Collecting passlib<1.8,>=1.6.5
Downloading passlib-1.7.2-py2.py3-none-any.whl (507 kB)
|████████████████████████████████| 507 kB 50.9 MB/s
Collecting sortedcontainers<2.1,>=1.5.4
Downloading sortedcontainers-2.0.5-py2.py3-none-any.whl (28 kB)
Collecting pyparsing<2.3,>=2.1.3
Downloading pyparsing-2.2.2-py2.py3-none-any.whl (57 kB)
|████████████████████████████████| 57 kB 4.2 MB/s
Collecting tornado<5.2,>=4.3
Downloading tornado-5.1.1.tar.gz (516 kB)
|████████████████████████████████| 516 kB 54.5 MB/s
Collecting pyasn1<0.5,>=0.3.1
Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
|████████████████████████████████| 77 kB 4.5 MB/s
Collecting ruamel.yaml<0.16,>=0.13.2
Downloading ruamel.yaml-0.15.100.tar.gz (318 kB)
|████████████████████████████████| 318 kB 61.5 MB/s
Collecting h2<4,>=3.0.1
Downloading h2-3.2.0-py2.py3-none-any.whl (65 kB)
|████████████████████████████████| 65 kB 3.2 MB/s
Collecting wsproto<0.12.0,>=0.11.0
Downloading wsproto-0.11.0-py2.py3-none-any.whl (17 kB)
Collecting brotlipy<0.8,>=0.7.0
Downloading brotlipy-0.7.0.tar.gz (413 kB)
|████████████████████████████████| 413 kB 45.1 MB/s
Collecting kaitaistruct<0.9,>=0.7
Downloading kaitaistruct-0.8.tar.gz (5.2 kB)
Collecting blinker<1.5,>=1.4
Downloading blinker-1.4.tar.gz (111 kB)
|████████████████████████████████| 111 kB 52.1 MB/s
Collecting six>=1.5.2
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting cffi!=1.11.3,>=1.7
Using cached cffi-1.14.2-cp38-cp38-manylinux1_x86_64.whl (410 kB)
Collecting asn1crypto>=0.21.0
Downloading asn1crypto-1.4.0-py2.py3-none-any.whl (104 kB)
|████████████████████████████████| 104 kB 57.2 MB/s
Collecting idna>=2.1
Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting hpack<4,>=3.0
Downloading hpack-3.0.0-py2.py3-none-any.whl (38 kB)
Collecting h11~=0.7.0
Downloading h11-0.7.0-py2.py3-none-any.whl (53 kB)
|████████████████████████████████| 53 kB 1.6 MB/s
Collecting pycparser
Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB)
Using legacy setup.py install for urwid, since package 'wheel' is not installed.
Using legacy setup.py install for pyperclip, since package 'wheel' is not installed.
Using legacy setup.py install for tornado, since package 'wheel' is not installed.
Using legacy setup.py install for ruamel.yaml, since package 'wheel' is not installed.
Using legacy setup.py install for brotlipy, since package 'wheel' is not installed.
Using legacy setup.py install for kaitaistruct, since package 'wheel' is not installed.
Using legacy setup.py install for blinker, since package 'wheel' is not installed.
Installing collected packages: click, six, pycparser, cffi, asn1crypto, idna, cryptography, pyOpenSSL, pyasn1, ldap3, urwid, pyperclip, certifi, hyperframe, passlib, sortedcontainers, pyparsing, tornado, ruamel.yaml, hpack, h2, h11, wsproto, brotlipy, kaitaistruct, blinker, mitmproxy, gitlab-apifuzzing-recorder
Running setup.py install for urwid ... done
Running setup.py install for pyperclip ... done
Running setup.py install for tornado ... done
Running setup.py install for ruamel.yaml ... done
Running setup.py install for brotlipy ... done
Running setup.py install for kaitaistruct ... done
Running setup.py install for blinker ... done
Successfully installed asn1crypto-1.4.0 blinker-1.4 brotlipy-0.7.0 certifi-2020.6.20 cffi-1.14.2 click-6.7 cryptography-2.3.1 gitlab-apifuzzing-recorder-1.6.33 h11-0.7.0 h2-3.2.0 hpack-3.0.0 hyperframe-5.2.0 idna-2.10 kaitaistruct-0.8 ldap3-2.5.2 mitmproxy-4.0.4 passlib-1.7.2 pyOpenSSL-18.0.0 pyasn1-0.4.8 pycparser-2.20 pyparsing-2.2.2 pyperclip-1.6.5 ruamel.yaml-0.15.100 six-1.15.0 sortedcontainers-2.0.5 tornado-5.1.1 urwid-2.0.1 wsproto-0.11.0