Manage ssh config on SRE dev machines

Background

As part of our SRE onboarding machine setup we configure SSH.

This SSH config is needed to properly proxy through bastion hosts in order to gain production access to chef-managed machines.

Problem

This config is a one-time setup. After this initial step, it is completely unmanaged. That means that as changes are made to the base config, there is no way to propagate them to dev machines.

As a result, there is a large amount of drift between configurations. Some of the existing configs have broken edge-cases, and getting the updates rolled out to everyone is a challenge.

Case in point: https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/merge_requests/446.

Proposal

We should introduce a process and tooling to manage this configuration. This will enable us to:

• effectively roll out changes to the SSH config
• clean up unused logic
• make it work more consistently and reliably
• feed improvements back so that everybody can benefit from them

Solution

New glsh ssh-config command: gitlab-com/runbooks!8834 (merged).