Migrate from gossip to ssb-standard-conn

TO DO

SSB Gossip Rewrite

Goals

Simplify (untangle the implementation)
Modularize (split it into 4+ modules)
Generalize (use multiserver addresses and never assume host:port shape)
Customization via distros (i.e. instead of using lots of config fields in ssb-config, make it easy for developers to recombine and package their own gossip plugin)
Small (each of the modules should be about 200 lines of code each, in contrast current ssb-gossip is 700+ LOC)

Non-goals

(Things that could be done, and would be nice, but are not yet an actual problem, and by doing these we may introduce a new class unknown of problems)

Replace atomic-file with another database
Generalizing this work to non-SSB contexts

Proposed design

SSB CONN (Connections over Numerous Networks) as a stack of modules:

graph BT
    ssb-conn-query --> ssb-legacy-conn
    ssb-conn-db --> ssb-legacy-conn
    ssb-conn-db --> ssb-conn-query
    ssb-conn-hub --> ssb-conn-query
    ssb-conn-staging --> ssb-conn-query
    ssb-conn-hub --> ssb-conn-staging
    ssb-conn-staging --> ssb-legacy-conn
    ssb-conn-hub --> ssb-legacy-conn
    
    subgraph ssb-gossip alternatives
       ssb-legacy-conn
       ssb-conn
    end

ssb-conn-db

Takes care of persisting data concerning connections.

Contract

Needs

Path where the local database should be stored
atomic-file

Provides

CRUD API, key-value style where string address is the key
multiserver-address validation of the given addresses upon insert
Simple query APIs such as "best peer(s) under certain criteria"
Possibility to store peers that don't have a public crypto identity (e.g. rooms)
API to listen to changes to the DB
- Important if a ssb-gossip plugin wants to always attempt connect upon DB insertion

ssb-conn-hub

Takes care of managing connections active during runtime.

Contract

Needs

pull-ping

Provides

API to connect to a specific peer (returns promise)
API to disconnect from a specific peer (returns promise)
API to list all currently connected peers (returns promise of array)
API to listen to changes to the set of currently connected peers
- like gossip.changes
API to enable/disable a mode (a type of peer)
API to listen to modes being switched on/off
API to live detect the status
- Offline: OFF (all connectivity modes are turned off)
- Standby: ON (some connectivity modes are on), DISCONNECTED, and IDLE
- Seeking: ON, DISCONNECTED, and BUSY
- Online: ON, CONNECTED to some

ssb-conn-utils

Provides common connection-establishing tactics and algorithms. Perhaps this should be a Wiki with many independent utils modules?

Contract

Needs

ssb-conn-db
ssb-conn-hub

Provides

API to reconnect all
API to select peers based on criteria
API to connect to one peer that matches a criteria
- For example, gossip/schedule.js::connect() function
- Perhaps this is just utils::select() followed by a hub::connect()?
API to connect with exponential capped backoff upon errors
Listen to Hub events (such as connect/disconnect) and store health stats and ping to the db
API to disconnect one peer that matches a criteria

ssb-example-gossip

Template project showing how to setup your own ssb-gossip. Draft:

function ssbCONN(connDB, connHub) {
    connHub.onStatusChange((status) => {
        onSomeEvent(() => {
            utils.connectToMany(someCriteria)
        })
    })
    
    connHub.onConnect(() => {
        if (connHub.amount > MAX) {
           utils.disconnectOne(someCriteria) 
        }
    })
    
    return pluginAPI
}

ssb-legacy-gossip

Mimics as much as possible the current ssb-gossip plugin.

Mapping the current gossip plugin

Features

Exports

add: sync
- insert a peer's address to the local gossip db
remove: sync
- remove a peer's address from the local gossip db
enable: sync
- Enable a type (local/global/seed) of connections to happen
disable: sync
- Disable a type (local/global/seed) of connections from happening
connect: async
- connect to a given address, AND add it to the local gossip db
peers: sync
- Fetch all currently connected peers
changes: source
- Listen to changes to set of currently connected peers
reconnect: sync
- Nuke current connections and try to connect to any peers
ping: duplex
- Used internally to measure latency and clock skew

Imports

has-network
- To detect whether the current network interfaces give us any connection beyond the local device
on-change-network
on-wakeup
ip
- To detect whether an IP address is private, or whether it is loopback
statistics
- To calculate stats on duration to connect
atomic-file
- As the database

Behavioral

Reconnect on network interface changes
- Observe network interfaces, and when they change, call reconnect
Reconnect on computer waking up from sleep mode
- Send heartbeats to self, and if some are missing, it means computer slept
Analyze good peers to connect to
- Based on ping stats
- Based on whether they have been responding to our attempts
On startup, read the SSB log for messages type "pub", and attempt connection
Schedule new connections upon startup
Schedule new connections periodically
Do not allow connections to exceed a maximum number, from config
Periodically save in-memory peers to the local gossip database
Scheduling connections always has a random delay
- To prevent deadlock dancing with a remote peer following the same periodic rythym
Some peer types (e.g. local) are not loaded from db
On incoming (expected) connection, setup ping, update db, setup teardown logic
All peers must have a key (public identity)

Course of action

Basically, create ssb-legacy-gossip and literally just import ssb-gossip as is, then gradually write the other pieces, replacing parts as we go. In other words gradual migration, not sudden rewrite. See the top of this issue for a task list.

The idea is to use ssb-legacy-gossip in Manyverse, and gradually rewrite the internals, always using ssb-legacy-gossip in production. Once ssb-legacy-gossip is fully rewritten and conforms to the previous behavior, we can start building ssb-standard-gossip with new behavior.

ssb-standard-gossip can introduce:

Impossibility to connect with a blocked peer
Connect based on hops distance (see %PyiK5ksJlFD+cZCj5OZo7o7OdZxf6LaopOiwK75Ib1s=.sha256)
Exponential capped backoff on a per node basis
- As a utility in ssb-conn-utils?
- As an option given to ssb-conn-hub::connect(opts)?
Configurability of ssb-standard-gossip through ssb-config (while allowing other ssb-gossip implementations to ignore or not those configs)
Etc

For reference, these are the instances where high-level who-to-connect logic is coded:

ssb-friends will replicate all friends https://github.com/ssbc/ssb-friends/issues/26
add local peers to gossip: https://github.com/ssbc/ssb-server/blob/master/plugins/local.js#L52-L63
ssb-replicate also has policy to always replicate local peers: https://github.com/ssbc/ssb-replicate/blob/master/legacy.js#L194-L200 it would be better if that was handled by local plugin, so that this happened when replicating on ebt also. The reason it's like this is because legacy replication predates this policy of separating how from what.
add pub messages to gossip: https://github.com/ssbc/ssb-gossip/blob/master/init.js#L15-L27
add address messages to gossip: https://github.com/ssbc/ssb-device-address/blob/master/index.js#L35-L64 (this should supercede pub messages)

References:

ssb-replicate versus ssb-ebt and how to accomodate the APIs

Edited Jul 10, 2019 by staltz