ETag Response Header
MERGE REQUEST
Overview
This MR sets the ETag response header for GET and HEAD made to /skynet/skylink/*skylink, which is the Skynet download route. Browsers will automatically supply the value received in this response header on consecutive calls to the server, if we find those values match, we can return a '304 Not Modified', instructing the browser to use its version of the data saving time and bandwidth.
The way you construct the ETag, basically your cache key, is obviously very important. Luckily you can easily invalidate if you mess up, as you can just change it slightly server-side. I decided to incorporate the following parts in the cache key:
- skylink
- request method (GET vs HEAD)
- path
- format
It's important we think about whether or not this is sufficient. Supplying the nocache query string parameter disables the 304 Not Modified response and will result in a regular download. Originally I hashed the entire URL object from the request, but that yields different ETags if you were to supply nocache=0 vs nocache=1, or if you would supply a timeout. Both are examples of behaviour we do not want.
We need this for HSN, Karol explains why:
For skylink urls, we report to browser (with a header) that it can cache it for 24 hours and that's ok because it's always unique (has skylink) - this way uniswap and other apps load almost instantly when loaded for the second time on the same browser.
Now with hns domains situation is different, user can update the hns domain record and request ie https://siasky.net/hns/uniswap-dex and expect to load a new version when reloading. This means we cannot tell browser to cache the response for 24 hours, it needs to make a request each time. This would mean it needs to download all the files every time we reload the browser but this can be improved by introducing etags. Etag in a simple case is a hash of the response body content and is sent to a browser in a header. Then when browser requests again for the same file, it starts loading the response headers first and compares the etag it had and etag it receives - in case they match, request is stopped and local content is loaded, otherwise it loads a whole response and updates the stored etag.
After this change is deployed, we can probably get rid of the 5' cache period we impose on the Nginx level.
Example for Visual Changes
N/A
Checklist
Review and complete the checklist to ensure that the MR is complete before assigned to an approver.
-
All new methods or updated methods have clear docstrings -
Testing added or updated for new methods -
Any new packages are added to Makefile and .gitlab-ci.yml -
API documentation updated for API updates -
Module README.md updated for changes to workflow -
Issue added to Sia-UI repo for new supporting features -
Changelog File Created
Issues Closed
N/A