Need to be able to Delete Corrupt SiaFile
PROGRAMMING TASK
Description of Task
Update the workflow of deleting a siafile
to succeed even if there is a read error.
Reason or Need for Change
Renters should always be able to delete a siafile
even if it is corrupted. Otherwise those files would stay on the network indefinitely and continue to cause issue with background services like the repair process.
Design / Proposal
Current Flow
The current work flow for deleting a siafile
is as follows:
- Open
siafile
(Disk Read) - Delete
siafile
- Close
siafile
We open the siafile
through the siafileset
in order to maintain thread safety, this is the only part of the process where the siafile
is read from disk. The deletion of the siafile
from disk removes it without reading from it. The Delete
method is a method of the siafile
and we also check if the siafile
was deleted by another thread which is another reason we need to open and access the siafile
.
Proposal
In thinking of how to change the work flow in order to enable the ability to delete a corrupt siafile
we do not want to loose the thread safety we have. So the main change is around how we handle opening the siafile
.
Currently the open
method of the siafileset
checks if the siafile
is in memory and if not it reads it from disk. If we create an OpenToDelete
method for the siafileset
and then a Delete
function to go along with the Delete
method of the siafile
, we can handle the case of a corrupt file better. The method would go something like the following:
func (sfs *SiaFileSet) OpenToDelete(siaPath modules.SiaPath) error {
// Acquire lock of SiaFileSet
// Check for SiaFile in memory
// If in memory, handle deletion the same as we do now.
// If Not in memory
// Create and apply delete wal update without opening the SiaFile.
// Holding the lock on the SiaFileSet should prevent other threads
// from opening the SiaFile while this is happening
}
If the siafile
is already in memory, then we shouldn't have any issue deleting it because the corruption must have occurred after it was read from disk, and the actual deletion does not read from disk again. If the siafile
is not in memory, we should prevent the siafile
from being opened again by holding the lock of the siafileset
and then delete the siafile
without opening it to avoid any error in reading the file due to corruption.