PARTIAL SUCCESS: Upgrade Geo multi-server installation from 12.9.10 to 12.10.12

Versions

Current: 12.9.10 Target: 12.10.12

Upgrade checklist

Preflight - upgrade

• Create a zoom meeting and schedule a recorded meeting
• Check if the Geo HA update instructions contain version specific changes
• Check if PostgreSQL is already the latest shipped version. If not, ensure PostgreSQL upgrade instructions are followed.
• Check if any upgrade warnings exist
• Verify that the Geo cluster is healthy pre-upgrade. For a no-downtime upgrade, remove deploy nodes from load balancers/stop sidekiq and run looping-pipeline to confirm test pass.
• Find the latest packaged version of GitLab that can be used for zero downtime upgrades
• Set up the looping test pipeline to run during the upgrade procedure.

Upgrade

• Retrieve a beverage of choice within a drinkable temperature range
• Join Zoom meeting and wait for arrival. Hit the record button
• Manually trigger the looping test pipeline to start running smoke tests before upgrading the primary site and before upgrading the secondary site (if failures happen during the primary site update)
• Perform upgrade steps described in latest documentation.
• During the upgrade process, monitor the looping test pipeline for any failures
• Record any issues encountered during the upgrade
• Verify cluster health post upgrade
• Verify PostgreSQL version is correct

Postflight

• Record the upgrade outcome as SUCCESS (upgrade with zero downtime), FAILED, PARTIAL SUCCESS (upgrade but with downtime or unconfirmed downtime)
• Open new issues and inform @nhxnguyen and @fzimmer to confirm next steps
• Create new issue for the next upgrade demo (the next versions) and assign to @nhxnguyen and @fzimmer
• Open an issue/MR to update Geo validation tests docs page (doc/administration/geo/replication/geo_validation_tests.md)