Skip to content

Remove Consensus Startup Code

David Vorick requested to merge ouch into master

The consensus package, prior to this PR, needed to scan the whole blockchain at startup so that it could put some change entries in memory. With a simple change that I should have made a while ago, I've removed the need to do that. Now, all modules subscribe using the PersistSubscribe function, which could really be renamed the Subscribe function at this point. Two methods of the consensus interface have been removed entirely, ConsensusChange and ConsensusSetSubscribe.

This should decrease startup time and also hopefully reduce memory usage at peak, though it's clear that Sia inefficiently uses memory in a lot of ways. Just like with the long startup time, I think that getting the memory usage back down is going to require a large number of smaller optimizations rather than a single large optimization.

I think I can point to IBD and inefficient block + transaction broadcasting as responsible for as much as half of the memory that siad uses, and is probably responsible for a lot of network consumption and performance problems during IBD. But in general I think we've been sloppy about keeping memory usage to a minimum, and probably the same with computation.

At this point, we're going to need to keep an eye on all of those things, as well as go back through our old code and make sure we are being efficient.

Merge request reports