Configure WAL-G's "wal-push" and "backup-push" to test Postgres backups creation with WAL-G on production
In https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/8903, we started to test WAL-G for both the creation and use of backups on staging.
After 3 weeks of successful testing (see Daily staging restore using WAL-G, from WAL-G staging archive
in https://ops.gitlab.net/gitlab-com/gl-infra/gitlab-restore/postgres-gprd/pipeline_schedules), it is time to start testing on a production replica.
-
Decide how replica will be chosen and how it will be configured using Chef (note that patroni-01 is the master in gprd right now). Such replica is to be marked as unavailable for failover. -
In addition to WAL-E, install WAL-G 0.2.14 to all Postgres instances (how it can be done: https://ops.gitlab.net/gitlab-com/gl-infra/gitlab-restore/postgres-gprd/blob/master/bootstrap.sh#L84). It is okay to have both WAL-E and WAL-G installed. -
Configure WAL-G. Use some new GCS bucket, different from the existing ones. -
on the chosen replica in gprd
, put WAL-G'swal-push
toarchive_command
, and setarchive_mode
to'always'
. -
Ensure that wal-push log does not have errors, but shows successful WAL-pushing and the new GCS bucket/folder is being filled with WALs. -
Configure daily backup-push
on the same replica (a cronjob similar to WAL-E'sbackup-push
cronjob on the master), to have full backups daily.
Additional TODOs:
Edited by Henri Philipps