Investigate adding PyPI support to the Package Registry
Problem to solve
As part of our goal to build a product that in 3 years will allow 90% of our customers to use GitLab for all of their package management needs, we must add support for common package manager formats, such as PyPI. Prior to adding support for a given package manager, we need to have a task of investigation of documentation, API endpoints, and other implementation requirements.
This issue is intended to detail the requirements for the PyPI Repository MVC and link to issues outside the scope of the MVC. It will be considered complete when the product developers have broken down the MVC into sub-issues and added them to the PyPI Repository epic.
Intended users
- Delaney (Development Team Lead)
- Sasha (Software Developer)
- Devon (DevOps Engineer)
- Sidney (Systems Administrator)
Further details
What is PyPI
The Python Package Index, or PyPI, is a vast repository of open-source Python packages supplied by the worldwide community of Python developers. The official index is available at https://pypi.org, and the site itself is maintained by the Python Software Foundation.
"PyPI" should be pronounced like "pie pea eye", specifically with the "PI" pronounced as individual letters, rather as a single sound. This minimizes confusion with the PyPy project, which is a popular alternative implementation of the Python language.
Use cases
- Users can use their GitLab project as a host for their PyPI Repository.
- Users can use their GitLab personal access token to authenticate to their project's PyPI repository.
- Users can configure their choice of PyPI client tools (setuptools, twine, distutils and pip) to create and configure a .pypirc file to reference packages stored in the repository manager.
- Users can install packages using
pip install
or from the CLI or GitLab CI/CD. - Users can upload packages to GitLab using
twine
- Users can search for packages in their PyPI repository with
pip search
Metadata (for the UI)
- Name
- Version
- Platform
- Summary
- Keywords
- Homepage
- Download URL
- Author
- Author email
- License
Vision
This contributes to our vision by providing a single application CI/CD experience for our customers and our users. And allowing us to better support Python developers.
What's not included in this investigation
- Conda which is especially popular with data scientists. We will evaluate adding support for this after our initial release of the PyPI repository.
Proposal
Investigate adding PyPI support to the GitLab Package Registry. Based on that investigation, identify a reasonable MVC that can be delivered in 1-2 milestones.
- Create, size and schedule a list of sub-issues and add them to the epic
- Get feedback from the community to ensure that we will support the most common use cases and workflows.
Links / references
- JFrog's PyPI Integration
- Sonatype's PyPI Integration
- PyPI API Docs
- PyPI-private project
- PyPI hosting your own index
- Devpi PyPI server and packaging/testing/release tool
- https://github.com/pypa/warehouse/issues/284
- Conversation with PyPI on twitter
Permissions and Security
The permissions should follow the same levels as all other package registries
Project Permissions: UI
Action | Guest | Reporter | Developer | Maintainer | Owner |
---|---|---|---|---|---|
Pull from Maven, NPM, Conan, NuGet, PyPI | x | x | x | x | |
Publish to Maven, NPM, Conan, NuGet, PyPI | x | x | x |
Project Permissions: API
Action | Guest | Reporter | Developer | Maintainer | Owner |
---|---|---|---|---|---|
List project packages (5) | x | x | |||
Get a project package | x | x | |||
List package files | x | x | |||
Delete a project package | x | x |
Group Permissions: API
Action | Guest | Reporter | Developer | Maintainer | Owner |
---|---|---|---|---|---|
[List the packages of a group | x | x |
Instance Level Permissions
Action | Guest | Reporter | Developer | Maintainer | Owner |
---|---|---|---|---|---|
Enable the Packages feature | x | ||||
Migrate local packages to object storage | x | ||||
Disable the Packages feature | x |
What does success look like, and how can we measure that?
- Success looks like we have a clear path and estimates to deliver an MVC