Add a setting to allow/disallow duplicate Maven package uploads
Release notes
Have you been using the GitLab Package Registry to publish your Java dependencies with Maven and/or Gradle? You may have run into a problem with publishing the same version multiple times. For Java packages, you expect to be able to publish duplicate snapshots, but not releases.
We've addressed this issue by adding a new group setting for the package registry that allows you to choose whether or not you'd like to allow or disallow duplicate Maven or Gradle uploads. By default, duplicates will not be allowed with the exception of snapshots. You can easily adjust this by navigating to your group->settings->packages & registries and enable duplicates or add an exception using regular expression.
After Maven and Gradle, we'll be doing the same for other package manager formats as well. We'll follow the standard set by the public registries.
Problem to solve
When using the GitLab Package Registry to upload packages, a user can upload the same Maven package name/version multiple times. The newest record will always be served when installing. The older records will only be accessible via the UI or API. This can result in the wrong dependency being installed and can introduce risk into the software development lifecycle.
Proposal
Offer a setting that will allow Admin to determine if they want to allow or disallow duplicate Maven packages to be published to their registry. By default, duplicate Maven uploads will be allowed. In addition, allow an Admin to add any additional exceptions using regex.
This table is a good reference for our current duplicate behavior, and it can be used to determine what the default value should be as we work through adding this setting for each package type.
Package type | Duplicates allowed? |
---|---|
npm | |
maven | |
NuGet | |
Conan | |
PyPI |
A word on Snapshots
A snapshot version in Maven is one that has not been released.
The idea is that before a 1.0 release (or any other release) is done, there exists a 1.0-SNAPSHOT. That version is what might become 1.0. It's basically "1.0 under development". This might be close to a real 1.0 release, or pretty far (right after the 0.9 release, for example).
The difference between a "real" version and a snapshot version is that snapshots might get updates. That means that downloading 1.0-SNAPSHOT today might give a different file than downloading it yesterday or tomorrow.
Usually, snapshot dependencies should only exist during development and no released version (i.e. no non-snapshot) should have a dependency on a snapshot version.
How to exclude SNAPSHOTS
For Maven SNAPSHOT artifacts the regular version is 0.9.3-SNAPSHOT
. This is what is published to the repo by Maven or Gradle, however, the artifacts are stored in the repository (and physically served later on) are named:
foo-bar-0.9.3-20201027.142837-1.jar
and for another build:
foo-bar-0.9.3-20201028.193111-1.jar
In other words, there is now SNAPSHOT keyword included. For example, check out this SNAPSHOT repository hosted by Sonatype Nexus.
Therefore, the exclusion rule has to operate on the logical name to make it work as expected.
Questions
-
✅ Should this setting be at the instance/group/project level? Can it be set at the instance level as a default and then inherited (but adjustable) at the group level?- We've decided to implement this at the group level to start.
-
✅ About 10% of Maven packages on GitLab.com are duplicated How will this setting impact those users? The same goes for GitLab Self-Managed instances.- This will not be a breaking change.
- What about Gradle? Do we need a separate issue/design/implementation?
- No, although the artifact (file) itself has a different name, in the database, we only store the common name and version.
Links
Testing
Test group settings for allowing/disallowing duplicate Maven packages