Skip to content

Add the API endpoint for the Maven dependency proxy

🎈 Context

We're working on the very first version of the dependency proxy for packages. See #407460 (comment 1373731852) for all the details from the technical investigation.

At the core, the concept is right simple. GitLab will act with as a proxy. Basically, users can pull packages through it and GitLab will be a pull-through cache.

Package Manager clients (npm, maven, ...) <-> GitLab <-> External Package Registry

Because, GitLab is in the middle (aka proxy) of the package transport, we can leverage the GitLab Package registry to use it as a cache. In other words, before contacting the external package registry, we can check the local project registry to check if the package is already there. If that's the case, we can return it directly.

The first iteration will only cover the Maven package format.

Add table for the dependency proxy packages set... (!120894 - merged) added the model class needed for the settings of the dependency proxy. Among other things, this model holds the url of the external package registry.

This MR is about adding the API endpoint to which maven clients ($ mvn, $ gradle or $ sbt) can point to and pull packages through it.

Being the very first of the API endpoint, we're going to add a basic version. The whole dependency proxy work is behind a feature flag = it is ok to have an endpoint that actually does nothing. The actual dependency proxy will take a few MRs to get right and so was out of the scope of this MR to keep this MR under reasonable size levels.

That doesn't mean that we don't want the basics covered:

  • The endpoint should have the very same parameters (and its validations) that the Maven Repository download endpoint.
  • For the permissions part, we want to follow the read_package permission. See the discussion here.
  • The endpoint should fetch the dependency proxy settings and check that it is enabled.
  • The endpoint should not be executed if packages or dependency_proxy are not enabled in the GitLab config.
  • Being part of a the GitLab Premium tier, the API class is in the EE side and we check the project license to assert that we are at least on the GitLab Premium.

See The dependency proxy API skeleton for Maven pac... (#410721 - closed)

🤔 What does this MR do and why?

  • Add a GET /api/v4/projects/:project_id/dependency_proxy/packages/maven/*path/:file_name route with all the necessary validations and checks.
  • Add the related specs.
  • Add a policy for the dependency proxy setting object to define the read_package permission. This one will follow the exact same rules than on project policy except for the "allow anyone" option.
  • One word on authentication, we need to support exactly the same tokens as the ones from the Maven registry download endpoint.

The API endpoint is gated behind a feature flag. Here is the rollout issue: [Feature flag] Enable packages_dependency_proxy... (#415218 - closed).

📺 Screenshots or screen recordings

n / a

How to set up and validate locally

  1. Make sure that you have a GitLab license set up in your local GDK as this MR changes are gated behind the GitLab Premium license level.
  2. Enable the feature flag in a rails console:
    Feature.enable(:packages_dependency_proxy_maven)
  3. DependencyProxy::Packages::Setting.create!(project: Project.find(1), maven_external_registry_url: 'http://test.sandbox', enabled: true)

We are now ready to test this.

Let's access the dependency proxy. The package name and file_name doesn't matter much as we don't have any logic that use them.

$ curl "http://<username>:<pat>@gdk.test:8000/api/v4/projects/1/dependency_proxy/packages/maven/foo/bar/1.2.3/foo.bar.1.2.3.pom" 
{"message":"202 Accepted"}

The request was accepted 🎉

Let's remove the setting in the rails console:

DependencyProxy::Packages::Setting.first.destroy!

Let's access the dependency proxy again:

$ curl "http://<username>:<pat>@gdk.test:8000/api/v4/projects/1/dependency_proxy/packages/maven/foo/bar/1.2.3/foo.bar.1.2.3.pom" 
{"message":"404 Dependency proxy setting Not Found"}

Setting not found

Other conditions can be played around:

  • No license in the GitLab install.
  • feature flag disabled.
  • packages disabled in the config.
  • dependency_proxy disabled in the config.

🏎 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by David Fernandez

Merge request reports