Package Registry - pypi support for Python compatibility tags

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

PEP 491 guidelines indicate that {python tag} should contain language implementation and version tag, for example 'py27', 'py2', 'py3'

The wheel filename is {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl

This is one of the "compatibility tags' which express the package's basic interpreter requirements and are detailed further in PEP 425

The customer tried to build multiple packages for Python 3.5, 3.6, 3.7, populating this part of the filename with 'py35', 'py36', 'py37'

There's more information in a forum post and GitLab team members can find out more in the ticket

Steps to reproduce

Copied from the forum post

  1. extension of the mypypipackage example from GitLab docs
  2. in my setup.py I am “baking” python into the wheel
import sys
import setuptools

# Set the required version of python.
__python_version__ = "=={}.{}.*".format(sys.version_info.major, sys.version_info.minor)

# Set python tag if building a wheel.
if "bdist_wheel" in sys.argv:
    if not any(arg.startswith('--python-tag') for arg in sys.argv):
        sys.argv.extend(['--python-tag', "py{}{}".format(sys.version_info.major, sys.version_info.minor)])

setuptools.setup(
    name="mypypipackage",
    version="0.1.3",
    author="tkornuta",
    description="A small example package",
    packages=setuptools.find_packages(),
    classifiers=[
        'Programming Language :: Python :: {}.{}'.format(sys.version_info.major, sys.version_info.minor),
        "License :: OSI Approved :: MIT License",
        "Operating System :: OS Independent",
    ],
    python_requires=__python_version__,
)
  1. set up the GitLab CI to build three wheels, for three different python versions/containers
python36:
  image: "registry.example.com/docker/python:3.6"
  script:
    - python -c "import sys; print('{}.{}.*'.format(sys.version_info.major, sys.version_info.minor))"
    - pip install twine
    - python setup.py bdist_wheel
    - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url  https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*

python37:
  image: "registry.example.com/docker/python:3.7"
  script:
    - python -c "import sys; print('{}.{}.*'.format(sys.version_info.major, sys.version_info.minor))"
    - pip install twine
    - python setup.py bdist_wheel
    - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url  https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*

python38:
  image: "registry.example.com/docker/python:3.8"
  script:
    - python -c "import sys; print('{}.{}.*'.format(sys.version_info.major, sys.version_info.minor))"
    - pip install twine
    - python setup.py bdist_wheel
    - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --repository-url  https://gitlab.example.com/api/v4/projects/<project_id>/packages/pypi dist/*
  1. the package registry should display three versions of the wheel
  2. when installing the package with the environment set for any of the three versions ..
pip install mypypipackage --index-url https://<personal_access_token_name>:<personal_access_token>@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
  • one will work
Looking in indexes: https://<personal_access_token_name>:****@@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
Collecting mypypipackage
  Downloading https://bbb/mypypipackage-0.1.3-py38-none-any.whl (1.6 kB)
Installing collected packages: mypypipackage
Successfully installed mypypipackage-0.1.3
  • the other two will not
Looking in indexes: https://<personal_access_token_name>:****@@gitlab.example.com/api/v4/projects/<project_id>/packages/pypi/simple
ERROR: Could not find a version that satisfies the requirement mypypipackage==0.1.3
ERROR: No matching distribution found for mypypipackage==0.1.3

Example Project

What is the current bug behavior?

The package repository does not return versions of the package for different python versions.

Summarizing @sabrams comments in slack:

  • the multiple packages published are identified by GitLab identifies as having the same name and version
  • all three wheels are stored as the same (duplicate) package
  • GitLab code allows publishing duplicates, the last uploaded one is always served up when requested
  • GitLab cannot differentiate between the three packages, one for each python version
  • The three packages are stored together in the GitLab registry, and regardless of configuration, the most recent upload will be returned.

What is the expected correct behavior?

Unique packages can be published using the language implementation and version tag specifically, and the compatibility flags in general.

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Possible fixes

Edited by 🤖 GitLab Bot 🤖