Add the API endpoint for the Maven dependency proxy
🎈 Context
We're working on the very first version of the dependency proxy for packages. See #407460 (comment 1373731852) for all the details from the technical investigation.
At the core, the concept is right simple. GitLab will act with as a proxy. Basically, users can pull packages through it and GitLab will be a pull-through cache.
Package Manager clients (npm, maven, ...) <-> GitLab <-> External Package Registry
Because, GitLab is in the middle (aka proxy) of the package transport, we can leverage the GitLab Package registry to use it as a cache. In other words, before contacting the external package registry, we can check the local project registry to check if the package is already there. If that's the case, we can return it directly.
The first iteration will only cover the Maven package format.
Add table for the dependency proxy packages set... (!120894 - merged) added the model class needed for the settings of the dependency proxy. Among other things, this model holds the url
of the external package registry.
This MR is about adding the API endpoint to which maven clients ($ mvn
, $ gradle
or $ sbt
) can point to and pull packages through it.
Being the very first of the API endpoint, we're going to add a basic version. The whole dependency proxy work is behind a feature flag = it is ok to have an endpoint that actually does nothing. The actual dependency proxy will take a few MRs to get right and so was out of the scope of this MR to keep this MR under reasonable size levels.
That doesn't mean that we don't want the basics covered:
- The endpoint should have the very same parameters (and its validations) that the Maven Repository download endpoint.
- For the permissions part, we want to follow the
read_package
permission. See the discussion here.- The challenge here is that we don't want to be impacted by https://docs.gitlab.com/ee/user/packages/package_registry/#allow-anyone-to-pull-from-package-registry, which is an option to "open" the
read_package
permission to Internet. - I don't think we have any other choice here than: creating a new policy based on the dependency proxy setting and define same rules for
read_package
except the "allow anyone option".
- The challenge here is that we don't want to be impacted by https://docs.gitlab.com/ee/user/packages/package_registry/#allow-anyone-to-pull-from-package-registry, which is an option to "open" the
- The endpoint should fetch the dependency proxy settings and check that it is enabled.
- The endpoint should not be executed if
packages
ordependency_proxy
are not enabled in the GitLab config. - Being part of a the GitLab Premium tier, the API class is in the
EE
side and we check the project license to assert that we are at least on the GitLab Premium.
See The dependency proxy API skeleton for Maven pac... (#410721 - closed)
🤔 What does this MR do and why?
- Add a
GET /api/v4/projects/:project_id/dependency_proxy/packages/maven/*path/:file_name
route with all the necessary validations and checks. - Add the related specs.
- Add a policy for the dependency proxy setting object to define the
read_package
permission. This one will follow the exact same rules than on project policy except for the "allow anyone" option. - One word on authentication, we need to support exactly the same tokens as the ones from the Maven registry download endpoint.
The API endpoint is gated behind a feature flag. Here is the rollout issue: [Feature flag] Enable packages_dependency_proxy... (#415218 - closed).
📺 Screenshots or screen recordings
n / a
⚙ How to set up and validate locally
- Make sure that you have a GitLab license set up in your local GDK as this MR changes are gated behind the GitLab Premium license level.
- Enable the feature flag in a rails console:
Feature.enable(:packages_dependency_proxy_maven)
-
DependencyProxy::Packages::Setting.create!(project: Project.find(1), maven_external_registry_url: 'http://test.sandbox', enabled: true)
We are now ready to test this.
Let's access the dependency proxy. The package name and file_name doesn't matter much as we don't have any logic that use them.
$ curl "http://<username>:<pat>@gdk.test:8000/api/v4/projects/1/dependency_proxy/packages/maven/foo/bar/1.2.3/foo.bar.1.2.3.pom"
{"message":"202 Accepted"}
The request was accepted
Let's remove the setting in the rails console:
DependencyProxy::Packages::Setting.first.destroy!
Let's access the dependency proxy again:
$ curl "http://<username>:<pat>@gdk.test:8000/api/v4/projects/1/dependency_proxy/packages/maven/foo/bar/1.2.3/foo.bar.1.2.3.pom"
{"message":"404 Dependency proxy setting Not Found"}
Setting not found
Other conditions can be played around:
- No license in the GitLab install.
- feature flag disabled.
-
packages
disabled in the config. -
dependency_proxy
disabled in the config.
🏎 MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.