Pre-batch items in separate buffers in rumbad, to reduce locking requirements
From BuildGrid/rumba!6 (comment 690731403)
At first I thought that this was the STL queue which is not thread safe, then realized we have a thread-safe queue implementation. https://gitlab.com/BuildGrid/rumba/-/blob/c4938d159541cf4f800bbb50d633fed1079c07de/rumbad/rumbad_queue.h
I wonder if the overhead here could be significant (grabbing and releasing lock for every single message).
Since we have our own queue implementation, it may make sense to "detect" that we have reached the batch size as we insert things, and potentially have a "backlog" of "buffers" of batch size? That way when we try to publish we can try to see if we have any buffers in the "backlog" to publish, without requiring the lock to the buffer everyone's writing to?
(And then we can define behavior in cases where the current "front" buffer is not full yet).