Skip to content

rewrite the whole upload process

David Vorick requested to merge test-cleanup into master

This is not the final version of the code, but what I've done here is completely thrown away the old, complex upload code and replaced it with some code that moves in a pretty linear fashion:

  1. find all the unfinished work in the whole renter, and assemble a queue (to be replaced when we build out our filemapper)
  2. pull chunks out of the queue 1 at a time, while at the same time making sure that any incoming files are being added to the queue
  3. allocate the memory, to make sure memory usage is properly throttled
  4. fetch the data for the chunk asynchronously
  5. pass the chunk+data to the workers, who have independent methods for dealing with the chunks.

There are still a bunch of TODOs, but the basic idea should function okay. The biggest missing thing is that we don't report back correctly if a worker fails to upload. Should be a simple change though.

Merge request reports