Design: Sectors Reference Counter
PROGRAMMING TASK
Description of Task
Add a reference counter file which accompanies each individual contract. This counter keeps track of the references to each sector within the contract and allows the renter to reuse (overwrite) sectors with zero references, thus reusing freed up space before the end of the current period.
Reason or Need for Change
When a renter uploads a file to a host and then stops using it before the end of the period they are de facto losing money because they need to keep paying the host for that space until the period ends. It would be a huge improvement for them to be able to reuse the already rented space in order to upload a new file. This is what we aim to enable.
Design / Proposal
The reference counter file is created and delete together with the contract file. It consists of a header section and a body. Write operations on the file are ensured to be ACID by using the WAL.
Header
The header holds metadata for the file - at this point that's only the version.
Body
The body contains a sequence of 16 bit unsigned ints - one per sector, representing the number of references for the given sector.
Open Questions & Next Steps
-
Start using the WAL -
Get rid of the in-memory slice to reduce ram usage -
Hook the refcounters to the rest of the code for non-production runs -
Sector counts are initialized with a predefined number which does not reflect the actual counts. We need to add an action that will scan the network, determine the actual counts and update the file. -
What happens when we overflow the uint16 counter? -
We should create ref counters for all existing contracts and trigger and populate them with correct values as soon as possible. We need to add migration code for this. -
Add triggers that will trim the garbage off a file contract before renewing it. Maybe one already exists and we can reuse it?