Skip to content

Separate protocol code from Contractor

Luke Champine requested to merge renter-reorg into master

The renter-host protocol is now fully encapsulated in renter/contractor/proto. The basic idea is that proto is a lightweight interface to the renter-host protocol; it should do minimal sanity checking, should be stateless (no logging/persistence), and generally assume that the caller knows what they're doing. This is the first step towards a major separation of concerns in the renter module. The various concerns are as follows:

Protocol tier:

  • Speak the renter-host protocol with hosts over a network connection
  • Provide simple abstractions that facilitate creating, revising, and renewing file contracts, as well as downloading stored data

Relationship tier:

  • Manage the user's allowance, enforcing spending limits and forming contracts of appropriate size and duration
  • Intelligently select hosts from the host database (Long-term, favor hosts who are faster, high collateral, etc.)
  • Allow the user to whitelist or blacklist hosts
  • Automatically renew file contracts before they expire
  • Manage a collection of potentially overlapping file contracts, distributing data amongst them

File tier:

  • Provide a file abstraction for securely and redundantly uploading a file
  • Encrypt files during upload and decrypt them during download
  • Use erasure coding to store files across multiple hosts
  • Monitor file health and reupload file data when necessary

Browser tier:

  • Provide a file system abstraction for managing files in a "Sia virtual drive"
  • Facilitate sharing and loading files via the .sia format

As of this PR, the protocol tier is handled by renter/contractor/proto, the relationship tier is handled by /renter/contractor, and the file+browser tier are handled by renter. The browser tier is relatively simple, so it may not need to be split from the file tier anytime soon.

I think this new organization will make it easier to incrementally improve the renter. It will also be easier for 3rd parties to hook into the renter at their desired level of abstraction. For example, someone who wants direct control over their redundancy could replace the file tier with their own implementation.

The downside to this approach is that it requires careful thought about how this tiered hierarchy should be expressed in the code base, especially with respect to the Renter interface and the API. Should all of the tiers' methods be crammed into one giant interface, or should each tier have its own interface? (Do all those interfaces need to live in modules, or can they live under renter?) What should be hard-coded, and what should be user-configurable? How will the various tiers be initialized? How should subscription be handled? etc.

So far, I've come up with two approaches. One is to replace the Renter interface with a struct, containing one field per tier. The API would then access the various tiers with e.g. srv.renter.Browser.Rename. Equivalently, Renter could remain an interface, with one method per tier. The other approach is for the Renter interface to only contain the Browser tier methods, plus a method that provides access to the File tier object. The File tier object would in turn provide access to the Relationship object, and from there the Protocol tier.

My preference is for the first approach, but I think there is more room for improvement. The fundamental problem is that various users will want to interact with the renter on various levels of abstraction. Granted, we aren't obligated to make things super friendly for those users; if they have to import the package directly instead of talking to an HTTP API, that's not the end of the world.

This code is definitely a work-in-progress in some areas. Perhaps it should be merged into a feature branch.

Merge request reports