consul service postgresql_service failing on db host
Summary
On provisioning HA setup using 10.4 consul service /opt/gitlab/bin/gitlab-ctl repmgr-check-master
fails with
ERROR: psql: FATAL: Peer authentication failed for user "gitlab-consul"
Steps to reproduce
Using 10.4.5 Provision 3 x consul node cluster Provision 1 x db and set as master
What is the current bug behavior?
Consul check fails
What is the expected correct behavior?
Check to pass return 0 exit code
Relevant logs
Relevant logs
# /opt/gitlab/bin/gitlab-ctl repmgr-check-master Error running command: /opt/gitlab/embedded/bin/psql -qt -d gitlab_repmgr -h /var/opt/gitlab/postgresql -p 5432 -c "SELE CT name FROM repmgr_gitlab_cluster.repl_nodes WHERE type='master' AND active != 'f'" -U gitlab-consul ERROR: psql: FATAL: Peer authentication failed for user "gitlab-consul"
Details of package version
Provide the package version installation details
10.4.5 on all nodes
Environment details
- Operating System: Ubuntu 16.04
- Installation Target, remove incorrect values:
- VM: Vagrant
- Installation Type, remove incorrect values:
- New Installation
- Is there any other software running on the machine: No
- Is this a single or multiple node installation? Multi
- Resources
- Consul CPU: 2
- Consul Memory total: 256MB
- DB CPU: 2
- DB Memory total: 1024MB
Configuration details
Provide the relevant sections of `/etc/gitlab/gitlab.rb`
external_url "http://app1.vagrant" roles ['postgres_role'] postgresql['listen_address'] = '0.0.0.0' postgresql['hot_standby'] = 'on' postgresql['wal_level'] = 'replica' postgresql['shared_preload_libraries'] = 'repmgr_funcs' gitlab_rails['auto_migrate'] = false consul['services'] = %w(postgresql) postgresql['pgbouncer_user_password'] = '*********************' postgresql['sql_user_password'] = '*********************' postgresql['max_wal_senders'] = 3 postgresql['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) repmgr['trust_auth_cidr_addresses'] = %w(127.0.0.1/32 10.6.0.0/24) consul['configuration'] = { bind_addr: "10.6.0.21", retry_join: %w(consul1.vagrant consul2.vagrant consul3.vagrant) }
Workarounds
Applied 2 changes
- Add GRANTS to gitlab_consul
sudo gitlab-psql -d gitlab_repmgr <<-EOF
GRANT SELECT, DELETE ON ALL TABLES IN SCHEMA repmgr_gitlab_cluster TO "gitlab-consul";
GRANT USAGE ON SCHEMA repmgr_gitlab_cluster TO "gitlab-consul";
\q
EOF
- Add line
local gitlab_repmgr gitlab-consul trust
to /var/opt/gitlab/postgresql/data/pg_hba.conf
This workaround is only good until the next gitlab-ctl reconfigure
which will overwrite those changes.
Edited by Alexander Strachan