Bounded queues for inter-thread communication in backend/gateway
Right now, both the gateway and backend use unbounded mpsc queues for inter-thread communication.
As @q3k writes in his !32 (merged) comment:
The unbounded mpscs here make me uneasy - if these are just used as a way to join gRPC's to/from Backend streaming with a channel/queue-like object inside the Gateway, then these should probably be unbuffered channels (ie. channels of length 0). Othewise, this has the potential to runaway and OOM any time comms get stuck, instead of handling things gracefully. And generally, depending on an 'unlimited' buffer introduces some uncertainty in between deployments of this code.
Otherwise, if we actually want to rely on some buffer here to allow for asynchronous (ie. best effort / eventually consistent) messaging to/from the Backend, then that buffer should IMO be explicitly sized, and behaviour on buffer overrun should be explicitly handled (even a crash/disconnect is fine, but something should happen instead of nothing for a long time and then a sudden OOM).
So, well, yes, we probably want to switch that to unbuffered channels. This should be a relatively simple change, but still requires reading the code and spending a few minutes to think whether that's a good idea.