Verify pgbouncer behavior when multiple masters are detected
Currently it attempts to shutdown pgbouncer when multiple masters are detected. This did not work in production. The end result being that the last known master stayed as the master in pgbouncer.
I, [2018-04-26T01:24:17.996968 #76661] INFO -- : Running: gitlab-ctl pgb-kill --pg-database gitlabhq_production --user gitlab-consul
E, [2018-04-26T01:24:22.723791 #76661] ERROR -- : STDERR: Error running command: GitlabCtl::Errors::ExecutionError
E, [2018-04-26T01:24:22.723945 #76661] ERROR -- : STDERR: ERROR: Password for user gitlab-consul: E, [2018-04-26T01:24:22.723988 #76661] ERROR -- : STDERR: psql: fe_sendauth: no password supplied
E, [2018-04-26T01:24:22.724026 #76661] ERROR -- : STDERR: /opt/gitlab/embedded/service/omnibus-ctl/lib/gitlab_ctl/util.rb:17:in `rescue in get_command_output'
: GitlabCtl::Errors::ExecutionError (GitlabCtl::Errors::ExecutionError)
E, [2018-04-26T01:24:22.724063 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/service/omnibus-ctl/lib/gitlab_ctl/util.rb:14:in `get_command_output
'
E, [2018-04-26T01:24:22.724115 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/service/omnibus-ctl-ee/lib/pgbouncer.rb:93:in `pgbouncer_command' E, [2018-04-26T01:24:22.724157 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/service/omnibus-ctl-ee/lib/pgbouncer.rb:128:in `kill'
E, [2018-04-26T01:24:22.724193 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/service/omnibus-ctl-ee/pgbouncer.rb:46:in `block in load_file'
E, [2018-04-26T01:24:22.724227 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/omnibus-ctl-0.5.0/lib/omnibus-ctl.rb:202:in
`block in add_command_under_category' E, [2018-04-26T01:24:22.724261 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/omnibus-ctl-0.5.0/lib/omnibus-ctl.rb:730:in
`run'
E, [2018-04-26T01:24:22.724295 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/omnibus-ctl-0.5.0/bin/omnibus-ctl:31:in `<t
op (required)>' E, [2018-04-26T01:24:22.724329 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/bin/omnibus-ctl:23:in `load'
E, [2018-04-26T01:24:22.724366 #76661] ERROR -- : STDERR: from /opt/gitlab/embedded/bin/omnibus-ctl:23:in `<main>'