Implement deferred writes to the fallback for with-cache
Before raising this MR, consider whether the following are required, and complete if so:
-
Unit tests -
Metrics -
Documentation update(s)
If not required, please explain in brief why not.
I don't immediately see any useful metrics here; we already have metrics for the duration of a write request.
This MR still needs documentation, and could maybe do with benchmarking with real workloads.
Description
Currently, writing to the !with-cache-storage blocks until the blob is written to both the cache layer and the fallback layer. This is desired behaviour in some circumstances, but for some combinations this adds an unnecessary wait.
This MR adds an optional flag which causes the commit_write
method
to finalise the fallback's write in a background thread, and return once
the blob is written into the cache layer. This removes the need for
writers to wait for writes to slow storage backends, such as S3.
Obviously, this raises issues in cases where the cache is small or (when using multiple BuildGrids) shared between instances, and so should only be used in particular circumstances. An example use case could be a large Redis cache with an S3 fallback. Writes to S3 are much slower than writes to Redis, and a clustered Redis deployment should be reliable enough to only encounter minimal issues.
Validation
Deploy a with-cache storage with defer-fallback-writes: yes
set, and run some test builds.
Issues addressed
Part of #344 (closed)