Maintenance Control Hosts
One significant risk to the success of the maintenance are issues local to those performing the work: laptops can fail, connectivity may drop, power may be lost. Any localized event taking place in the middle of any step in the maintenance procedure will yield an unknown state in the process as connections into the environment get dropped.
In order to minimize this risk, we need to setup hosts in the environment from where the maintenance is executed. These are not bastion hosts, but hosts within the environment that have access to all necessary components during the maintenance. The primary purpose of said hosts is to run shared screen
sessions (or appropriate alternatives). Loss of connectivity to this hosts will result in detached screen
sessions (vs dead ones). It also allows at least two members of the team to share the sessions in real time.
The hosts should be configured so that everything necessary to execute changes in the environment is available locally.