Hostdb should always scan hosts we have contracts with
Currently, the strategy that the hostdb uses for scanning means that certain hosts that we are using in our contracts aren't guaranteed to be scanned.
We need to set up some extra code to make sure that the hosts we've selected for contracts are always getting scanned.
I don't want to extend the hostdb to be able to distinguish between them though, because that's a point where we need to keep the contractor synchronized with the hostdb, which could be difficult. Instead, we should either have the contractor added to the hostdb, or we should just have the contractor able to call the hostdb directly to issue a scan. I haven't yet decided which I prefer.
If the contractor is calling the hostdb directly, it'd need to do it during threadedContractMaintenance
or similar. If we always do it during threadedContractMaintenance
that might result in these hosts being scanned too frequently, the biggest downside being consumption of user resources.
The alternative method would be to add a function like "UsedHosts" to the contractor, then add an interface to the modules package "HostdbSubscriber" or something that has "UsedHosts" as a function. The contractor would call "hostdb.Subscribe" inserting itself the way that other modules insert themselves as subscribers to the consensus set. Then the hostdb, each time it does a scan, would call "UsedHosts" which dynamically fetches the list of hosts that the contractor is actively watching. Then we guarantee that all hosts are being checked every scan.
I should also add for context that at some point we're going to extend the scans in the contractor to include occasionally requesting a random piece of data from its hosts / contracts. This is to make sure that the host hasn't sneakily put the data in cold storage, and to penalize the host early if the host isn't storing the data or otherwise isn't storing the data at the desired access time. We'll probably want to add that as a separate process though, as doing a piece download is a much more expensive operation than just issuing a scan. And, it's a process that makes a lot more sense / is a lot less expensive if we've added partial downloads. (Though we'd still want to occasionally request a full piece to make sure that the host can give us sustained throughput on short notice)