Investigate: Dependency Proxy to include packages pulled from remote registries

Problem to solve

With the GitLab Dependency Proxy, you can proxy and cache container images hosted on Docker Hub, so that you can reduce your reliance on external dependencies and improve your build times.

The user interface for the Dependency Proxy is available at the group level and provides a copyable URL for using the proxy and lists the number of blobs currently stored in the cache. However, it does not currently display which image tags have been fetched, whether or not they are in the cache and how often they are used. This makes it difficult for the user to understand how effective this feature is and to validate that the correct image tags are being used.

In the future, the Dependency Proxy will work similarly with packages pulled from remote registries, such as npmjs.com, maven-central, or Artifactory.

We need to ensure that a user understands what items are being pulled through the proxy. They also need to be able to see any relevant metadata, such as package details or pipeline/commit data.

Intended users

User experience goal

As users begin to adopt the Dependency Proxy, the UI makes it clear that the feature is working as expected and helps the user to understand which images have been cached and when.

Proposal

  • As a Developer, when I navigate to my group's Dependency Proxy, I need to see which image tags have been fetched from Docker Hub and when so that I can validate that the correct image was used and troubleshoot when something has gone wrong.
  • As a Developer, when I am looking at which image tags have been cached using the Dependency Proxy, I'd like to see meta data for the image, so I can understand how it was built.
  • When a user lands on the Dependency Proxy main UI, they can click on the npmjs.org(default) remote registry to view the details. That detail view will include a list of packages pulled from the default. From there, we could display which packages have been pulled, which of those are cached (and for how long), and a warning if any of those show security warnings.
  • (beyond the MVC) As an Admin, When I am trying to understand how my team uses the Dependency Proxy, I need to know how often image tags are fetched from Docker Hub, how often they are pulled from the cache and the ratio of those two metrics.

Further details

  • As discussed in #241639 (comment 413250100), currently only the blobs are cached when an image is pulled, not the manifest. This will likely need to be resolved so that we can display meta data in the UI.
  • As discussed in this issue #215393 (closed) the design of the dependency proxy UI should also include an ability to flag possible security vulnerabilities being pulled in through the cache. So when an image tag is requested from Docker Hub, we also check against the known vulnerabilities database and include a warning in the UI.

Image Metadata

  • Name
  • Tags
  • Copyable install commands
  • Downloaded date
  • Last used
  • Size
  • OS/Arch

Package metadata

Links / references

Edited by Tim Rizzi