Add a second pgbouncer instance on each host

Our current database load balancing configuration uses DNS A records for service discovery.

We are currently working to change the application to use DNS SRV records, but until that change is in place, the port for secondary pgbouncer/postgres database connections is fixed (since A records do not include a port)

This means we can effectively run one pgbouncer per IP.

Since we're currently experiencing saturation on pgbouncer, one of the ideas that has been floated is to add a second virtual IP to each patroni machine, and then bind one pgbouncer instance to each IP.

We will also need to make sure that one of the instances also binds to the loopback interface, to avoid other possible issues.

I would consider this a tactical stopgap solution to alleviate current pain, rather than a strategic route forward.

@stanhu @msmiley - with the work you've been doing looking at the cause, do you think this would help or hinder out effort?

cc @clefelhocz1

Assigning to @glopezfernandez @Finotto for further onward assignment.

Edited Aug 22, 2019 by Andrew Newdigate