Hosts are dropping unexpectedly from the hostdb
BUG REPORT
In summary, I am finding unexpectedly that the hostdb of an freshly synced node shows more active and online hosts than an "aged" node, synced months ago. I have found this happens because hosts returning online after a long offline period are not re-scanned again, and are totally absent from the hostdb. I can't discard hosts are dropping from the db also from other issues
Expected Behavior
Once a host goes offline, I expected its entry to remain on the hostdb (as not active, indeed) so if it comes back online at a later time point, the periodical scans finds it again online. As a result, I expected the hostdb of a freshly synced node to be identical to the hostdb of an old node.
Observed Behavior
Last week the owner of the host siastore.servebeer.com:9982 (pubkey: ed25519:1b0feb79256da28cd4b683400c090c5e5260dd786ab8c4f5c2169f13db8e7c22
) let me know he reconnected his host after 1 month offline, but his host was not showing in SiaStats even after 1 week online. I found that a host with this pubkey was not present on the hostdb of any of my 3 nodes, not even as an offline host. This host was definitely present on the hostdb of my nodes months ago, as it was being benchmarked by my SiaStats system (https://siastats.info/hosts?=102). It was showing online, however, for a 4th Sia node I synced from the scratch very recently. I suggested the user to announce the host again, to see if that could force the nodes to scan it for the hostdb, and it worked. This suggests me that hosts that become offline for a long time period are completely deleted from the hostdb.
Probably connected to this, I realized this morning, after replacing one of the 3 Sia nodes I use for SiaStats with a new one synced from the scratch, that it was able to detect 23 hosts that were once in my hostdb and my SiaStats database months ago. None of these hosts are in the hostdb of the old nodes. This new Sia node I synced is in the same datacenter as the old one, so this can't be the result of IPs being banned or inaccessible. These are the pubkeys of some of these hosts that are only visible for the newly synced node:
ed25519:e853f5ad2d9d5d3aae7ea2dec39e1c1747640f8fa714a9e344000ad48f6fdc46
ed25519:fe57fd909237a3b0d24739bd517024cc974570f9b3229cca76dbdb699f83dd32
ed25519:c91b9371de135a5321d6aca8e0e352ea1d6ed5b67120f52f44c6f8df03c4dd93
ed25519:e209c594656bf395956f5c9c3fb5cf2e6de5d6342f5805ea61495da9e1808061
ed25519:8e8ed34020d3c0818c2070c974effbf4ed04d9cdb32a23ce2b869de143473067
I don't know if this is a bug or it is intentional, so hosts under certain circumstances are deleted permanently from the hostdb. If this is the intentional, I find it problematic for two reasons:
-
Hosts that have been offline accidentally for a long period are not always re-announcing when going back online, so the network is losing the ability of using them again, unless the renter syncs the client from the scratch.
-
Network monitors can't track effectively these returning/dropped hosts. As an example, 195.182.153.249:9982 is a high quality host with impressive network speeds that is now showing in SiaStats as online just because I synced my "EU01" Sia node from the scratch (https://siastats.info/hosts?=945). This host does not exist in the db of my other two nodes and it is showing offline for SiaHub.
If dropping hosts from the DB can't be corrected because it is a necessary feature, could it be possible to add an API endpoint or siac
command that forces to re-scan the blockchain for host announcements so users can periodically recover returning hosts?
Thanks in advance.
How to reproduce it (as minimally and precisely as possible)
Comparing the hostdb of a Sia node that has been online for several months with another one freshly synced can reveal online hosts in the second one not even showing as offline on the first one.
Environment
- Sia version: 1.4.1.1
- OS: Ubuntu