Support multiple network pollers
As part of the scheduler overhaul, we should also support multiple network poller threads, as a single thread (as used now) can get overloaded. The way I envision to work is roughly as follows: we span N network threads, and P process threads. Each process thread is assigned an network poller/thread in a round-robin fashion. When performing a non-blocking operation that would block, the socket is registered with the network poller associated with the current process thread. For the same of simplicity, both N and P default to the number of CPU cores, with N being limited to 127 threads.
To make this work, sockets also have to be changed: instead of storing a boolean that indicates they have been registered with a poller at some point, they store the index to the poller/thread as an i8
. A value of -1
indicates the socket hasn't been registered before. When registering a socket we check this value and re-register (if needed) the socket with the corresponding poller. If no value is set, we pick the poller associated with the current process thread.
An alternative to this is to ditch the registered flag entirely, and explicitly unregister sockets in the network poller when we're done with them. This removes the need for extra state per socket, but it's not clear yet what the cost of doing this is on the polling side. Personally I prefer taking this approach, as it simplifies the process of registering sockets.