Support publishing Composer package to one central GitLab project
Context
When comparing GitLab's Package offering to solutions like Artifactory or Nexus, a key missing feature is virtual registries. A virtual registry allows you to group many other repositories behind a single URL, to make distributing packages easier at large organizations.
GitLab does not yet support this feature. But, as a workaround, we suggest to customers that they use a single project to store all of their packages. This allows the customer to simplify their workflows and still use a single URL.
Problem to solve
We can't use an empty project to host a set of composer packages as described in https://docs.gitlab.com/ee/user/packages/workflows/project_registry.html. This is understandable. When a composer package is pushed or better said "registered", the endpoint will inspect the git tags of the given project but in this case, the project is empty.
Proposal
Update the API endpoint for the publish action so that a project_id can be passed. This way, users will be able to tell the API: "hey my composer package is on git tag X but the tag is actually on project P". That project_id will then need to be persisted so that when the package is pulled, the backend knows that this package is referencing a tag that is in an other project than the current one.
Further details
Another way to solve this would be to create a package file that is the archive of the target git tag. When the publish API endpoint is called, it should enqueue a background job that will pull the git tag from the target project and create a package file with the corresponding archive. When the package is pulled, it's just a matter of reading the package file and return it.
This would the load on Gitaly as the package file would act as a cache for the archive file. As in any cache, we would need to define when to remove that package file. The only way here is to add a callback to the git tag operations endpoints. When a tag is destroyed, we should destroyed any package file linked to it. This is a more complex solution but the approach is re-usable for other package types, namely, Go packages.
Documentation
- Update the documentation be register vs. publish a package which is more accurate and less confusing.