Image scaler binary with pure golang lib
Supersedes !578 (closed).
With this latest implementation to image scaling we are following up to complexity concerns with the seccomp based solution.
This implementation has the following properties:
- purely Go
- uses Go stdlib for decoding,
disintegration/imagingfor resizing and as anencodewrapper (which itself is merely a thin wrapper around stdlib encoders) - it incorporates fixes recommended by AppSec such as failing fast when content does not match content-type determined by the app, as well as limits the maximum file size to 250kB
- it compiles a separate binary, which interfaces with Workhorse through env, stdin, stdout
Regarding file size restrictions, those are based on heuristics, but my rationale was as follows:
- we know that the average avatar we currently store is ~50kB (compressed)
- during recent experiments in production with the
gmscaler, we saw that the average file size that was requested was ~90kB - we have a restriction in place in Rails that avatars must not exceed 200kB in size on disk
- I "added" 25% = 50kB as leeway
- we know that ~85% of our avatars are PNGs
- due to the use of
zlib, my understanding is that PNGs are more susceptible to compression bomb attacks than JPEGs - in a local benchmark, I found that a 300x300 PNG compressed at different quality levels (i.e. compression levels) always deflated to about 500kB of memory (images were anywhere between 180-250kB in size on disk, i.e. on average roughly our max allowed upload size)
- with a max procs setting of 100 currently, this would mean no more than 50MB consumed for image scaling on a given node at any time
- this sounded reasonable, hence I chose this threshold.
Edited by Matthias Käppler