Implement conditional GETs for image resizer
In our first iteration for image scaling, we did not consider caching to reduce scope of the MVC.
While the implementation is fast in production, we verified that we can reduce load on GitLab significantly by implementing a conditional GET strategy. We decided to use timestamps i.e. If-modified-since
and Last-modified
instead of etags.
Unlike with send-url
or send-file
, we cannot entirely rely on either the object storage provider or Golang serve*
helpers to handle this for us, since on top of not sending down a response payload in case a rescaled image is considered fresh, we also want to prevent forking into the scaler to begin with, which happens before we connect the scaler output stream with the client response stream.
We accomplish this by:
- for URL addressable files: copy any existing
Last-Modified
header from OS response over to client and compare withIf-modified-since
from request - for path addressable files: compare file
ModTime
withIf-modified-since
from request
If the validator passes, we return early by writing out a 304 Not Modified
.
This implementation is partially based on what Golang does internally in https://github.com/golang/go/blob/master/src/net/http/fs.go; I slightly modified some of those functions to better suit our use case. Unfortunately they are not exported and only accessible indirectly through e.g. ServeFile
or ServeContent
, but for the above mentioned reasons we cannot use those, since they would require us to first fork into the scaler to obtain a stream Reader
. I retained the Golang copyright notice.