Support multiple Kafka clusters
A nice to have feature would be for the proxy to communicate with multiple Kafka clusters. This would allow a deployment of the websocket proxy to be the "single point of entry" towards Kafka infrastructure inside a secured network, reducing both complexity and potential attack surface.
Thoughts on implementation details:
- Configuration will have to be provided for > 1 Kafka cluster.
- The proxy will require a "primary" cluster for its internal topics, etc.
- Clusters need to be identified with an unique ID inside the proxy config.
- Should have an endpoint to retrieve information on which clusters are available through the proxy.
- API will require an additional argument so the clients can specify which cluster to target.
- It might be a good idea to add an endpoint that can list topics, with info on which target a topic is located on. However, this requires some thought with regards to ACLs and security considerations.
This above list will evolve as the thought process matures...
Edited by Knut Petter Meen