Skip to content

Use the faraday follow redirects middleware in the registry clients

Steve Abramsrequested to merge
10io-registry-clients-faraday-params-custom-encoder into master

🍭 Context

The rails backend interact with the container registry to get information and data about container images and tags.

On tags, the rails backend extracts the created_at timestamp to return it on APIs (and display it on frontend). Getting the created_at timestamp involves getting the config data first. The config data is basically a json structure where there is a created key.

The config is stored as a blob on the Container Registry. Rails will thus need to:

  1. pull/download the config data first
  2. JSON decode it
  3. Read the appropriate field

In (1.), the Container Registry can choose to send a redirect response. That response will point to an object storage location. Rails thus need to handle that redirect.

Recently, we rolled out a change in the Container Registry to use the Google Cloud CDN for object storage locations. This means that the redirect location can point to a Google Cloud CDN.

We learned the hard way that an url in Google Cloud CDN can end with a signature (because it's a signed url, similar to signed AWS S3 urls). The signature can end with a trailing =. Example:

  • https://redirect.to.here?Foo=Bar&Signature=66dyddgqweg12b43431wd=

Rails uses faraday for all the interactions with the Container Registry. faraday has a built in params encoder. That encoder will thus transform our example into:

  • https://redirect.to.here?Foo=Bar&Signature=66dyddgqweg12b43431wd%3D

Now, guess what happens when the Google Cloud CDN receives %3D instead of =? Yes, 💥. It will deny the request.

Since rails can't get the config structure, it can't read the creation timestamp and ultimately will use nil as created_at for tags. This nil will get translated into Time.zone.now on the UI. This leads to a super confusing UX where all tags are Published just now.

The above is exactly #353244 (closed).

💡 Solution Design

Following redirects is a basic feature that http clients such as faraday usually have.

Fortunately, we have a middleware that fill our needs here.

More context in !81056 (comment 851205352).

Since this change is for the blob download function of the registry client, we're going to gate this change behind a feature flag. We can easily break several features in this function doesn't work properly. Here is the rollout issue: #353291 (closed)

🔬 What does this MR do and why?

  • For the faraday used to download blobs, use the FaradayMiddleware::FollowRedirects middleware.
  • Create or update the related specs.

🖼 Screenshots or screen recordings

feature flag disabled

Conditions created_at returned properly?
GCS (no CDN) ✅
GCS (with CDN) ❌ (that's the typebug) See this comment
AWS ✅

feature flag enabled

Conditions created_at returned properly?
GCS (no CDN) ✅
GCS (with CDN) ✅ See this comment
AWS ✅

🗼 How to set up and validate locally

The way to test this is to enable the registry in GDK but then use the container registry project to build a binary with object storage support.

  1. Enable the registry in GDK.
  2. $ gdk reconfigure && gdk restart.
  3. Stop the GDK registry : $ gdk stop registry
  4. Update <gdk_root>/registry/config.yml with: 
    storage:
  # filesystem:
  #   rootdirectory: /var/lib/registry
  gcs:
    bucket: dfernandez-packages
    keyfile: /Users/david/Downloads/dfernandez-5494dd2c-2bda264f293e.json
  5. Checkout the container registry project
  6. Build the binary with the support for the storage driver you want: 
    $ GO_BUILD_FLAGS='-tags include_gcs' make binaries
  7. Start the registry with: 
    $ /bin/registry serve <gdk_root>/registry/config.yml

Now you're ready to test the scenarios of the previous section. You will still need to have the cloud resources (object storage bucket and API keys) at hand.

🚥 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by David Fernandez

Merge request reports