Skip to content

Implement conditional GETs for image resizer

Matthias Käppler requested to merge mk/image-resizer-conditional-get into master

Refs gitlab#273404 (closed)

In our first iteration for image scaling, we did not consider caching to reduce scope of the MVC.

While the implementation is fast in production, we verified that we can reduce load on GitLab significantly by implementing a conditional GET strategy. We decided to use timestamps i.e. If-modified-since and Last-modified instead of etags.

Unlike with send-url or send-file, we cannot entirely rely on either the object storage provider or Golang serve* helpers to handle this for us, since on top of not sending down a response payload in case a rescaled image is considered fresh, we also want to prevent forking into the scaler to begin with, which happens before we connect the scaler output stream with the client response stream.

We accomplish this by:

  • for URL addressable files: copy any existing Last-Modified header from OS response over to client and compare with If-modified-since from request
  • for path addressable files: compare file ModTime with If-modified-since from request

If the validator passes, we return early by writing out a 304 Not Modified.

This implementation is partially based on what Golang does internally in https://github.com/golang/go/blob/master/src/net/http/fs.go; I slightly modified some of those functions to better suit our use case. Unfortunately they are not exported and only accessible indirectly through e.g. ServeFile or ServeContent, but for the above mentioned reasons we cannot use those, since they would require us to first fork into the scaler to obtain a stream Reader. I retained the Golang copyright notice.

Edited by Matthias Käppler

Merge request reports