Investigate replacing rabbitmq with Redis streams
With Redis 5.0, the stream data type is now available: https://redis.io/topics/streams-intro
I think it should be feasible to replace rabbitmq with streams, which would let me simplify the site infrastructure a bit more. This will take some investigation and work though - some of the pieces will include:
- Write a new "bridge" service to replace pg-amqp-bridge that listens for postgresql
NOTIFY
and adds them to redis streams - Replace the
PgsqlQueueConsumer
class with one that reads from streams instead for the consumers to inherit from.
This article includes some postgresql/streams pieces that might be useful to reference: https://brandur.org/redis-streams