Skip to content

Use the Dependency Proxy with private GitLab groups

Steve Abrams requested to merge 11582-dependency-proxy-private-projects into master

🔍 What does this MR do?

🌲 Background

The dependency proxy is a feature that allows users to pull and cache public container (docker) images through a group-level route. By caching commonly used images (think about FROM node in a CI pipeline), pipelines can be sped up and connection problems with docker hub can be avoided.

The problem is these image pulls occur outside of GitLab through the docker client. The user runs a command like

docker pull gitlab.com/groupname/dependency_proxy/containers/alpine:latest

and the docker client will make a set of requests to gitlab.com. If the group is private, before we allow the user to run such a command, we need some way of authenticating them within GitLab before we proxy any requests and cache any images associated to their group.

🍎 How we authenticate users

The docker client uses an authentication sequence where it first checks if it needs to be authorized to make a request to the "registry" (which is gitlab.com in this case). The registry will respond telling it how to request a token, and the client will then request a token.

This occurs when the user runs the command docker login gitlab.com.

Note: It is important to realize we are not talking about logging into the GitLab container registry, which would be docker login registry.gitlab.com. We are in effect logging into the dependency proxy, which is an artificial registry where we proxy requests to hub.docker.com on the user's behalf.

Once the token is saved by the client, any future requests will use the token. When the user logs in using their credentials or personal access token, we can return a JWT that encodes their user_id, and in future requests, we can find the user from the token, and check their permissions against the group the request is being made to.

This is what the docker login command sequence that this MR implements looks like:

sequenceDiagram
  autonumber
  participant C as Docker CLI
  participant R as GitLab (Dependency Proxy)

  Note right of C: User tries `docker login gitlab.com` and enters username/password
  C->>R: GET /v2/
  Note left of R: Check for Authorization header, return 401 if none, return 200 if token exists and is valid
  R->>C: 401 Unauthorized with header "WWW-Authenticate": "Bearer realm=\"http://gitlab.com/jwt/auth\",service=\"registry.docker.io\""
  Note right of C: Request Oauth token using HTTP Basic Auth
  C->>R: GET /jwt/auth
  Note left of R: Token is returned
  R->>C: 200 OK (with Bearer token included)
  Note right of C: original request is tested again
  C->>R: GET /v2/ (this time with `Authorization: Bearer [token]` header)
  Note right of C: Login Succeeded
  R->>C: 200 OK

We are able to use the existing jwt_controller, which is used for authenticating with the regular GitLab container registry, and we just have set up a new service: DependencyProxyAuthenticationService for building and returning the token when it is requested from the jwt_controller#auth route.

🤔 ...so what does this MR do?

  • Adds a controller to return 401 unauthorized and 'realm/service' information when users try to login (route /v2).
  • Adds a new authentication service to jwt_controller for use with the dependency proxy that returns a token with the user_id in it.
  • Updates the existing dependency proxy controller to check for this new token and find the user that will be used for authorization checks.
  • Removes the front end code that displayed a message saying you cannot use the dependency proxy with private groups.
  • Updates the dependency proxy docs.
  • Places the feature behind an enabled by default feature flag just as a safety measure in case we need to turn it off.
    • Note: the feature flag is instance level because the /v2 endpoint does not contain any information to identify a group.

☑ Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team

Related to #11582 (closed)

Edited by Steve Abrams

Merge request reports