Write tooling to do fleet upgrades

In order to facilitate a sustainable, consistent, and repeatable process that we can use with minimal up-front effort, or time spent re-engineering the same solutions every time, we need to begin automating the upgrade process and keep track of the specific steps for orchestrating the changes within the environment.

This issue will be used to coordinate the development of scripts, ansible playbooks, etc. required to facilitate upgrading VM instances to a new OS version. As these can/will vary by service, we should focus on a high-level process that is generic as possible, with service-specific nuances accounted for within appropriate stages. Ideally we will have both the orchestration of the high-level process and the per-service/per-node steps automated; one idea is to create one or more pipelines for the high-level process, with custom scripts executed via per-service/per-environment jobs in each stage, via child pipelines, or some combination of both.

Definition of done

  1. Document the high-level process, end-to-end
  2. Create a project or space within an existing project for source code and pipeline configs
  3. Setup credentials and configure/identify where the pipelines will be run
Edited by Craig Barrett