Investigate adding PyPI support to the Package Registry
Problem to solve
As part of our goal to build a product that in 3 years will allow 90% of our customers to use GitLab for all of their package management needs, we must add support for common package manager formats, such as PyPI. Prior to adding support for a given package manager, we need to have a task of investigation of documentation, API endpoints, and other implementation requirements.
This issue is intended to detail the requirements for the PyPI Repository MVC and link to issues outside the scope of the MVC. It will be considered complete when the product developers have broken down the MVC into sub-issues and added them to the PyPI Repository epic.
- Delaney (Development Team Lead)
- Sasha (Software Developer)
- Devon (DevOps Engineer)
- Sidney (Systems Administrator)
What is PyPI
The Python Package Index, or PyPI, is a vast repository of open-source Python packages supplied by the worldwide community of Python developers. The official index is available at https://pypi.org, and the site itself is maintained by the Python Software Foundation.
"PyPI" should be pronounced like "pie pea eye", specifically with the "PI" pronounced as individual letters, rather as a single sound. This minimizes confusion with the PyPy project, which is a popular alternative implementation of the Python language.
- Users can use their GitLab project as a host for their PyPI Repository.
- Users can use their GitLab personal access token to authenticate to their project's PyPI repository.
- Users can configure their choice of PyPI client tools (setuptools, twine, distutils and pip) to create and configure a .pypirc file to reference packages stored in the repository manager.
- Users can install packages using
pip installor from the CLI or GitLab CI/CD.
- Users can upload packages to GitLab using
- Users can search for packages in their PyPI repository with
Metadata (for the UI)
- Download URL
- Author email
This contributes to our vision by providing a single application CI/CD experience for our customers and our users. And allowing us to better support Python developers.
What's not included in this investigation
- Conda which is especially popular with data scientists. We will evaluate adding support for this after our initial release of the PyPI repository.
Investigate adding PyPI support to the GitLab Package Registry. Based on that investigation, identify a reasonable MVC that can be delivered in 1-2 milestones.
- Create, size and schedule a list of sub-issues and add them to the epic
- Get feedback from the community to ensure that we will support the most common use cases and workflows.
Links / references
- JFrog's PyPI Integration
- Sonatype's PyPI Integration
- PyPI API Docs
- PyPI-private project
- PyPI hosting your own index
- Devpi PyPI server and packaging/testing/release tool
- Conversation with PyPI on twitter
Permissions and Security
The permissions should follow the same levels as all other package registries
Project Permissions: UI
|Pull from Maven, NPM, Conan, NuGet, PyPI||x||x||x||x|
|Publish to Maven, NPM, Conan, NuGet, PyPI||x||x||x|
Project Permissions: API
|List project packages (5)||x||x|
|Get a project package||x||x|
|List package files||x||x|
|Delete a project package||x||x|
Group Permissions: API
|[List the packages of a group||x||x|
Instance Level Permissions
|Enable the Packages feature||x|
|Migrate local packages to object storage||x|
|Disable the Packages feature||x|
What does success look like, and how can we measure that?
- Success looks like we have a clear path and estimates to deliver an MVC