Skip to content

Host + Renter overhaul

Luke Champine requested to merge renter into master

At long last, we have erasure coding, parallel uploads and downloads, and file contract revisions. But I'm not terribly proud of some of this code; a lot of it is a bit shaky, and needs thorough review (especially contract negotiation + revision -- I'm pretty confident I'm not doing signatures correctly, because I can't figure out where to supply the secret keys).

Definitely missing from this PR:

  • Host + renter API updates, as described in #723 (closed). This includes adding json tags.
  • Compatibility is broken for .sia files, as well as host+renter persistence files and host RPCs. At the very least, hosts should be able to provide old files and submit old proofs, and renters should be able to retrieve old files. This may be tricky and/or ugly. We should find out what users are willing to put up with.
  • Revisions are never submitted to the blockchain. Both the host and renter need to submit the revision before WindowStart, and or immediately in the event of negotiation failure. (works for now, but will need modification when resumability is introduced)
  • The host and renter probably need to save some information (to disk) that they currently aren't. I'm fairly sure the renter needs to save the unlock conditions for each file contract, but there may be others. (Will be implemented later, to enable resumable uploads)
  • Revisions need miner fees. (not immediately)
  • Revisions do not account for changes in host settings
  • Lots of testing. Current coverage is renter: 43.9%, host: 17.2% (ouch!)

Finally, the upload and download algorithms need more work. The upload algorithm is (I believe) close to optimal in terms of speed, but may impact redundancy. The download algorithm is still terrible; I haven't come up with anything satisfactory yet. To make things easier, I've included commented-out statistics in TestErasureUpload and TestErasureDownload, so you can get immediate feedback on any modifications you make to the upload and download algorithms. Also, the testHost object should be modified to disconnect (either randomly or at a specific time) so that we can ensure such disconnects won't break uploads or downloads.

I think that's it. We're almost there, fellas. Just needs some polishing and testing, and then the real fun begins...

fixes #159 (closed) and fixes #642 (closed).

Merge request reports