Engineering Owner: @dbalexandre ### Introduction When performing a failover, systems administrators need to use different tools depending on the underlying architecture. When using a single-server Geo installation, the `gitlab-ctl promote-to-primary-node` command can be used; however, multi-server installations don't support this command yet. When using HA/multi-server two other commands need to be used * `gitlab-pg-ctl promote` on a PG node * `gitlab-rake geo:set_secondary_as_primary Additionally, systems administrators need to manually edit `gitlab.rb` on each server. This can add up to several dozens of machines depending on the reference architecture. ### Problem to solve Currently, `gitlab-ctl promote-to-primary` can only be run on a single-node Geo instance but does not support multi-node/HA installations (or handles external PG gracefully). ### Proposal **CAVEAT:** For the first iteration, we are focusing on utilising a single command, orchestration is out of scope 1. Implement functionality in `gitlab-ctl promote-to-primary` so that it can detect the type of node it is run on e.g. single-node, postgres, gitaly etc. 1. Handle external PG by dsiplaying a warning when no omnibus postgres is detected ### Permissions and Security `sudo` is required ### Documentation https://docs.gitlab.com/ee/administration/geo/disaster_recovery/index.html#promoting-a-secondary-node-with-ha needs to be updated ### Availability & Testing Should be tested manually in GCP and in staging ### What does success look like, and how can we measure that? * A systems administrator can use a **single** command to promote all node types from secondary to primary. ### What is the type of buyer? Premium and Ultimate