Add a size-differentiated storage provider
Before raising this MR, consider whether the following are required, and complete if so:
-
Unit tests -
Metrics -
Documentation update(s)
If not required, please explain in brief why not.
Any relevant metrics are likely more useful in the underlying storages rather than the wrapper.
Description
This is a wrapper around two or more other storages, used to store blobs in a different storage based on their size. This allows reserving a faster, more limited storage such as the LRU in-memory cache for small blobs, falling back to a less constrained storage for large blobs.
Validation
Deploy a BuildGrid using this storage with a small enough size that some blobs will not be in the first storage, for example:
storages:
- !lru-storage &lru-storage
size: 2048M
- !disk-storage &disk-storage
path: /home/adam/cas
- !size-differentiated-storage &cas-storage
size-limited-storages:
- max-size: 1K
storage: *lru-storage
fallback: *disk-storage
Then run a test build with that BuildGrid.
Issues
Part of #345 (closed) and related to #344 (closed)
Edited by Adam Coldrick