[PG HA] Database node fails on first reconfigure
According to the docs in https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/3151,
after configuring the consul nodes, we are configuring database nodes.
After specifying the configuration, doc says we need to run reconfigure. The first reconfigure fails because the database does not exist:
execute[Add pgbouncer auth function] action run
[execute] psql: FATAL: database "gitlabhq_production" does not exist
================================================================================
Error executing action `run` on resource 'execute[Add pgbouncer auth function]'
================================================================================
Mixlib::ShellOut::ShellCommandFailed
------------------------------------
Expected process to exit with [0], but received '2'
---- Begin output of /opt/gitlab/bin/gitlab-psql -d gitlabhq_production -c ' CREATE OR REPLACE FUNCTION public.pg_shadow_lookup(in i_username text, out username text, out password text) RETURNS record AS $$
BEGIN
SELECT usename, passwd FROM pg_catalog.pg_shadow
WHERE usename = i_username INTO username, password;
RETURN;
END;
$$ LANGUAGE plpgsql SECURITY DEFINER;
REVOKE ALL ON FUNCTION public.pg_shadow_lookup(text) FROM public, pgbouncer;
GRANT EXECUTE ON FUNCTION public.pg_shadow_lookup(text) TO pgbouncer;
' ----
STDOUT:
STDERR: psql: FATAL: database "gitlabhq_production" does not exist
---- End output of /opt/gitlab/bin/gitlab-psql -d gitlabhq_production -c ' CREATE OR REPLACE FUNCTION public.pg_shadow_lookup(in i_username text, out username text, out password text) RETURNS record AS $$
BEGIN
SELECT usename, passwd FROM pg_catalog.pg_shadow
WHERE usename = i_username INTO username, password;
RETURN;
END;
$$ LANGUAGE plpgsql SECURITY DEFINER;
REVOKE ALL ON FUNCTION public.pg_shadow_lookup(text) FROM public, pgbouncer;
GRANT EXECUTE ON FUNCTION public.pg_shadow_lookup(text) TO pgbouncer;
' ----
Ran /opt/gitlab/bin/gitlab-psql -d gitlabhq_production -c ' CREATE OR REPLACE FUNCTION public.pg_shadow_lookup(in i_username text, out username text, out password text) RETURNS record AS $$
BEGIN
SELECT usename, passwd FROM pg_catalog.pg_shadow
WHERE usename = i_username INTO username, password;
RETURN;
END;
$$ LANGUAGE plpgsql SECURITY DEFINER;
REVOKE ALL ON FUNCTION public.pg_shadow_lookup(text) FROM public, pgbouncer;
GRANT EXECUTE ON FUNCTION public.pg_shadow_lookup(text) TO pgbouncer;
' returned 2
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/pgbouncer_user.rb
29: execute 'Add pgbouncer auth function' do
30: command %(/opt/gitlab/bin/gitlab-psql -d #{database} -c '#{pgbouncer_auth_function}')
31: user account_helper.postgresql_user
32: not_if { pg_helper.has_function?(database, "pg_shadow_lookup") }
33: only_if { node.default['gitlab']['pgbouncer']['auth_query'].eql?(node['gitlab']['pgbouncer']['auth_query']) }
34: end
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab-ee/recipes/pgbouncer_user.rb:29:in `from_file'
execute("Add pgbouncer auth function") do
action [:run]
retries 0
retry_delay 2
default_guard_interpreter :execute
command "/opt/gitlab/bin/gitlab-psql -d gitlabhq_production -c ' CREATE OR REPLACE FUNCTION public.pg_shadow_lookup(in i_username text, out username text, out password text) RETURNS record AS $$\n BEGIN\n SELECT usename, passwd FROM pg_catalog.pg_shadow\n WHERE usename = i_username INTO username, password;\n RETURN;\n END;\n $$ LANGUAGE plpgsql SECURITY DEFINER;\n\n REVOKE ALL ON FUNCTION public.pg_shadow_lookup(text) FROM public, pgbouncer;\n GRANT EXECUTE ON FUNCTION public.pg_shadow_lookup(text) TO pgbouncer;\n'"
backup 5
returns 0
user "gitlab-psql"
declared_type :execute
cookbook_name "gitlab-ee"
recipe_name "pgbouncer_user"
not_if { #code block }
only_if { #code block }
end
Platform:
---------
x86_64-linux
/cc @ibaum
Edited by Marin Jankovski