Investigate replacing rabbitmq with Redis streams
With Redis 5.0, the stream data type is now available: https://redis.io/topics/streams-intro
I think it should be feasible to replace rabbitmq with streams, which would let me simplify the site infrastructure a bit more. This will take some investigation and work though - some of the pieces will include:
- Write a new "bridge" service to replace pg-amqp-bridge that listens for postgresql
NOTIFYand adds them to redis streams
- Replace the
PgsqlQueueConsumerclass with one that reads from streams instead for the consumers to inherit from.
This article includes some postgresql/streams pieces that might be useful to reference: https://brandur.org/redis-streams