Ensure `gitlab-ctl reconfigure` has recently run on all hosts prior to failover
Per 2018-07-24 attempt
As part of the failover, we ensure the secondary is running the same code as the primary, and has the same configuration (more or less). Applying these things requires gitlab-ctl reconfigure
to run on the machines.
On some classes of machine - particularly pgbouncer and postgresql - automatically running reconfigure
is disabled.
We manually ran reconfigure
on a set of nodes during the failover to correct a problem that arose. In so doing, the pgbouncer authentication broke, causing the GCP site to stop working.
To flush out these kinds of potential issues prior to the migration, I suggest we ensure gitlab-ctl reconfigure
has been recently run on all these hosts. If there are problems with doing so, this will flush them out beforehand.
Ive given this a slightly higher weight than 1 as I think chances are good that we will flush a few problems out, and this will need reaction.
/cc @jarv