Skip to content

Gracefully handle absent storage folders

David Vorick requested to merge host-sf-wiping into master

Wanted some comments and ideas on the overall design. This PR (assuming it builds - didn't check b/c 5am) adds code to gracefully handle the situation where siad loads and a storage folder is not available.

Today the behavior is to delete that storage folder from memory, including all of the metadata. This means the host will lose that storage folder if they load siad at boot and the host starts up before the storage folder is available. v1.1.2 and earlier would fail silently, dropping all historic data as well and then putting the new data in the wrong location.

New behavior basically will still report the storage folder to the user, and will still retain all metadata, but will return some gross error statistics, and it will also prevent all operations involving the storage folder (reads + writes, namely).

I could also add some sort of background check which regularly checks whether the storage folder has been added and then logs an error if not, and marks the storage folder as available (opening all needed file handles) if it does? That part isn't implemented yet.

This PR could have more safety, though not quite sure how best to achieve it. Mostly this moves the host in the direction of robustness.

Side note: right now we have a json bitfield being encoded as a series of uint64s. Json prints numbers as a string, meaning that we end up with things taking 20+ bytes each (plus formatting overhead - burdens the CPU) when they should really only be taking up 8 bytes each (and ideally no formatting overhead). Not sure if there's an easy encoding fix here but @lukechampine it'd make the host run a lot smoother, and also it'd make the disks last longer. A better solution would not have us writing out a giant bitfield in JSON every half second, but that's a more difficult change to make.

Merge request reports