Configuring pgbouncer, pgb-notify fails when customised directories have been used
Summary
Configure gitlab.rb
with customised directories for pgbouncer
roles ['pgbouncer_role']
pgbouncer['data_directory'] = '/apps/gitlab/pgbouncer'
pgbouncer['databases_json'] = '/apps/gitlab/pgbouncer/databases.json'
pgbouncer['databases_ini'] = '/apps/gitlab/pgbouncer/databases.ini'
When reconfigure is run it fails with
[execute] The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node
The command
/opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer
is failing and is reliant on the existence of /var/opt/gitlab/pgbouncer
.
Workaround is to create this directory, no files are stored under it.
Steps to reproduce
Configure gitlab.rb
with customised directories for pgbouncer
roles ['pgbouncer_role']
pgbouncer['data_directory'] = '/apps/gitlab/pgbouncer'
pgbouncer['databases_json'] = '/apps/gitlab/pgbouncer/databases.json'
pgbouncer['databases_ini'] = '/apps/gitlab/pgbouncer/databases.ini'
What is the current bug behavior?
Command
/opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer
fails on the initial reconfigure
run as well as on subsequent uses.
/opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer
The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node
What is the expected correct behavior?
pgb-notify
to run when customised pgbouncer directories are used.
Relevant logs
Relevant logs
+;service_name = pgbouncer +;job_name = pgbouncer + +;; Read additional config from the /etc/pgbouncer/pgbouncer-other.ini file +;%include /etc/pgbouncer/pgbouncer-other.ini +%include /apps/gitlab/pgbouncer/databases.ini - change mode from '' to '0600' - change owner from '' to 'gitlab-psql' - change group from '' to 'gitlab-psql' * execute[reload pgbouncer] action run (skipped due to only_if) * file[databases.json] action create - create new file /apps/gitlab/pgbouncer/databases.json - update content in file /apps/gitlab/pgbouncer/databases.json from none to 44136f --- /apps/gitlab/pgbouncer/databases.json 2021-05-04 06:41:22.763406000 +0000 +++ /apps/gitlab/pgbouncer/.chef-databases20210504-7353-zai0fi.json 2021-05-04 06:41:22.763406000 +0000 @@ -1 +1,2 @@ +{} - change mode from '' to '0600' - change owner from '' to 'root' - change group from '' to 'gitlab-psql' * execute[generate databases.ini] action run [execute] The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node [execute] The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node [execute] The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node [execute] The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node================================================================================ Error executing action `run` on resource 'execute[generate databases.ini]' ================================================================================ Mixlib::ShellOut::ShellCommandFailed ------------------------------------ Expected process to exit with [0], but received '1' ---- Begin output of /opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer ---- STDOUT: The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node STDERR: ---- End output of /opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer ---- Ran /opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer returned 1 Resource Declaration: --------------------- # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/pgbouncer/recipes/enable.rb 76: execute 'generate databases.ini' do 77: command lazy { 78: <<~EOF 79: /opt/gitlab/bin/gitlab-ctl pgb-notify \ 80: --databases-json #{node['pgbouncer']['databases_json']} \ 81: --databases-ini #{node['pgbouncer']['databases_ini']} \ 82: --hostuser #{node['pgbouncer']['databases_ini_user']} \ 83: --hostgroup #{account_helper.postgresql_group} \ 84: --pg-host #{node['pgbouncer']['listen_addr']} \ 85: --pg-port #{node['pgbouncer']['listen_port']} \ 86: --user #{node['postgresql']['pgbouncer_user']} 87: EOF 88: } 89: action :nothing 90: not_if do 91: node['consul']['watchers'].include?('postgresql') && 92: File.exist?(node['pgbouncer']['databases_ini']) 93: end 94: retries 3 95: end 96: Compiled Resource: ------------------ # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/pgbouncer/recipes/enable.rb:76:in `from_file' execute("generate databases.ini") do action [:nothing] default_guard_interpreter :execute command "/opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer\n" backup 5 declared_type :execute cookbook_name "pgbouncer" recipe_name "enable" domain nil user nil retries 3 not_if { #code block } end System Info: ------------ chef_version=15.14.0 platform=ubuntu platform_version=20.04 ruby=ruby 2.7.2p137 (2020-10-01 revision 5445e04352) [x86_64-linux] program_name=/opt/gitlab/embedded/bin/chef-client executable=/opt/gitlab/embedded/bin/chef-client
Running handlers: There was an error running gitlab-ctl reconfigure:
execute[generate databases.ini] (pgbouncer::enable line 76) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of /opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer
STDOUT: The directory /var/opt/gitlab/pgbouncer does not exist. Please ensure pgbouncer is configured on this node STDERR: ---- End output of /opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer
Ran /opt/gitlab/bin/gitlab-ctl pgb-notify --databases-json /apps/gitlab/pgbouncer/databases.json --databases-ini /apps/gitlab/pgbouncer/databases.ini --hostuser root --hostgroup gitlab-psql --pg-host 0.0.0.0 --pg-port 6432 --user pgbouncer returned 1
Running handlers complete Chef Infra Client failed. 4 resources updated in 20 seconds
Details of package version
Provide the package version installation details
Noted by a customer in version 13.8.8, still exists in 13.11.3
Environment details
-
Operating System:
"Ubuntu 20.04.2 LTS"
-
Installation Target, remove incorrect values:
- Bare Metal Machine
-
Installation Type, remove incorrect values:
- New Installation
-
Is this a single or multiple node installation? single
Configuration details
Provide the relevant sections of `/etc/gitlab/gitlab.rb`
roles ['pgbouncer_role'] pgbouncer['data_directory'] = '/apps/gitlab/pgbouncer' pgbouncer['databases_json'] = '/apps/gitlab/pgbouncer/databases.json' pgbouncer['databases_ini'] = '/apps/gitlab/pgbouncer/databases.ini'