Skip to content

p2p: limit connections to 800 and add smother curve for limits

pietro requested to merge nomadic-labs/tezos:abate@p2p_disconnected_take2 into master

This MR is work in process to reduce the memory consumption associated to a reckless use of the option --connections. In particular it limits this parameter to 800 and try to smooth the curve that is used to compute the p2p layer limits max_known_points and max_known_peer_ids .

At the moment the curve is liner f(x) = (x * 8, x / 4 * 3) and for a large x the limits are too large.

The idea is to smooth this curve to better adapt to the number of connections and reduce the number of known ids and peers kept in memory.

This is the behavior of a node (master) running on chartagenet when providing --connections 500

Screenshot_2020-06-18_14-21-48

Considering the function above, for large number of connections (> 500), the table of known points and know ids can grow too large.

I propose to change the max and target thresholds as follow :

the max_peer_table size defined by the function

  • max = ec + ec * c - (ec / mc) where ec is the number of expected connections and mc is the maximum number of connections
  • 3 - (ec / mc) is the multiplier bounded between [2 - 1]
  • c is a constant >= 2 that can be adjusted to double or triple the size of the peer_table. The maximum number of connections is bounded to 800.

in other words the size of peer table is proportional to the expected connections but its growth is bounded by the max_connections the target size is 80% of the max size

for a --connection XX you get the following computed values

New (connections, max,target) :

50,175,140
100,350,280
200,700,560
300,1050,840
400,1400,1120
500,1750,1400
600,2100,1680
700,2450,1960
800,2800,2240

Old :

50,400,300
100,800,600
200,1600,1200
300,2400,1800
400,3200,2400
500,4000,3000
600,4800,3600
700,5600,4200
800,6400,4800
Edited by pietro

Merge request reports