Skip to content

Add environment variables to override backup/restore DB settings

Stan Hu requested to merge sh-pgbouncer-bpass-config into master

In the latest versions of PostgreSQL, using pg_dump on a PgBouncer connection can cause a full GitLab outage. This happens because pg_dump clears the search path and explicitly sets the schema for every SQL query it runs. From https://github.com/postgres/postgres/blob/110d81728a0a006abcf654543fc15346f8043dc0/src/bin/pg_dump/pg_dump.c#L3276-L3288, pg_dump clears the search_path with the following:

SELECT pg_catalog.set_config('search_path', '', false)

When PgBouncer is used in transaction pooling mode, these connection settings persist and cause queries made by Rails to fail since the public schema is not searched.

Currently there is no way to tell whether a connection is using PgBouncer (https://github.com/pgbouncer/pgbouncer/issues/249), so there is no way to prevent pg_dump from running.

To avoid causing an outage, we provide admins with a way to override the database settings for the backup and restore task via environment variables:

  • GITLAB_BACKUP_PGHOST
  • GITLAB_BACKUP_PGUSER
  • GITLAB_BACKUP_PGPORT
  • GITLAB_BACKUP_PGPASSWORD
  • GITLAB_BACKUP_PGSSLMODE
  • GITLAB_BACKUP_PGSSLKEY
  • GITLAB_BACKUP_PGSSLCERT
  • GITLAB_BACKUP_PGSSLROOTCERT
  • GITLAB_BACKUP_PGSSLCRL
  • GITLAB_BACKUP_PGSSLCOMPRESSION

Relates to:

Edited by Stan Hu

Merge request reports