[META] Failovers for Postgres work out of the package
Staging: configure and running via chef:
-
spin up 2 new dbs - https://gitlab.com/gitlab-com/infrastructure/issues/2228 -
sync dump via wale to one - https://gitlab.com/gitlab-com/infrastructure/issues/2229 -
setup repmgr - https://gitlab.com/gitlab-com/infrastructure/issues/2230 -
setup repmgrd - https://gitlab.com/gitlab-com/infrastructure/issues/2173 -
pgbouncer with consul - https://gitlab.com/gitlab-com/infrastructure/issues/2422
Production: configure and running via chef:
-
repmgrd + consul + pgbouncer - https://gitlab.com/gitlab-com/infrastructure/issues/2581
For reference:
- repmgr will handle the replication side of things (slaves and masters). It is a handy tool to enable us to easily manage who will be promoted to master, get a slave in sync with the cluster, and allow a slave to follow a (new) master
- repmgrd is a daemon running in the background which can detect master failures, and take steps such as promote a slave, and can even be configured to reconfigure pgbouncer to automatically connect to the new master.
- pgbouncer accepts and proxies the database connections in once single place, letting the connections to the master be configured here vs. on each server, with features such as pausing connections, moving them to a new server, etc. all without needing the applications to be reconfigured.
Edited by Jason Tevnan