Create a command line tool that pauses/resumes all replication on the secondary

Current state: After a successful POC, we determined that a first iteration would be a CLI tool that can handle pausing of all data on the secondary.

Problem to solve

Currently, pausing replication on the Primary for Geo does only pause replication for repositories and files. We should enhance the existing pause logic and include database replication. This is only possible on the Secondary Geo node.

Intended users

• Sidney (Systems Administrator)

Further details

#36217 (closed) evaluated if it was possible to pause database replication and resume again.

Proposal

1. Create gitlab-ctl geo:pause or similar that pauses all replication
2. Create gitlab-ctl geo:resume or similar that resumes all replication
3. As the secondary node is on read-only mode, to pause the node (enable=false) we need to make an API request to primary node.
4. Documentation

• Update api to allow nodes to disable themselves
• Add service and rake task to make api call to disable nodes
• Add gitlab-ctl command to pause wall and call rake task - omnibus-gitlab!4331 (merged)
• Add documentation on pausing secondaries from their consoles - !33113 (merged)
• Change front end button to indicator with a link to documentation on pausing nodes

gitlab-ctl or gitlab-rake may be right, not sure which.

Documentation

We need to update documentation to explain what "pausing" entails and what the possible risks are. We also need to describe the differences between the UI on the primary and the CLI tool on the secondary.

Testing

Pausing and restarting should be tested thoroughly because it could break the Geo setup

What does success look like, and how can we measure that?

The gitlab-ctl command can be successfully run on a secondary node.

What is the type of buyer?

• Premium

Links / references