altssh.gitlab.com unavailable after the front-end change
Following infrastructure#1747 `altssh.gitlab.com` became unavailable.
I have found the root cause in the old `lb10` and `lb11` not being able to load the new configuration.
The fix was to simply update `haproxy.cfg` with the new nodes. The outage ended today at 07:22 UTC.
This is what we need to avoid this from happening again in the future:
- Review our monitoring for altssh.
- Move the service to a new set of load balancers in ARM.
- Delete everything from the old platform to avoid confusion.
- As a more general thing, don't leave services/nodes half migrated.
issue