Meta: evaluate how this project compares to alternatives
This is pretty much a meta issue and was triggered by the question: "Do we need this project in the first place"?
The initial motivation for it was to have an easy way to use Kafka. I've noticed that it was way too much trouble for us to be able to use Kafka in a way provided authenticated TLS connections without going through the pain which is setup it on Kafka or AWS.
I wanted something as simple as using Basic HTTP auth over HTTPS. As it seems Kafka does not support that, I decided to have a websocket layer on top of it, to prevent using the Kafka protocol itself, which seems to be based on TCP.
A weekend plus some hours of hacking and this project was born. Direct to production. No time to think too much.
And it's been in production for months now, working quite well so far.
But.... did we need to create it in the first place? Quickly googling I could not find anything in the wild that suited our needs, but maybe I give up googling too quickly.
Googling again, I found this article [1] which lists some alternatives [2], [3], [4], and all the open source ones seem to be very old, abandoned many years ago and, we the article said, POC not ready for production (of course, they are selling their own product, so beware of biases).
So I ask myself again: do we need it?
So far the question has been yes, as the alternatives would be using some very old Java written proxies (beware all the FactoryFactories!) or pay for a complex business solution.
This project has been created to address very specific needs, which are ours, and I don't know though how it'd look if it needed to aim to be the "best in town". It'd probably fail miserably (especially regarding performance!).
My goal for this issue is to evaluate whether we could, instead of having this in-house implementation, reuse/buy something else.
Any thoughts @tcanabrava and @AleCalafange?
[1] https://ably.com/topic/websockets-kafka
[2] https://github.com/VictorGil/transfers_websockets_service