Use terraform for managing the fleet
Currently we have at least 3 different cloud providers, each with it's own tweaks and knows that make it really hard to maintain and automate.
For example, @jnijhof knows how to build VMs without spending a whole day in Azure because he is extremely used to it, but even he had to recently mention a specific tweak about caching when adding blobs because he got bit by this when building these VMs.
Also, since we are using digital ocean, AWS and Azure as providers it means that we need to know all these craziness and specific features from all these providers, and also, there is no way of having a glance at the whole infrastructure in a single place => we are wasting resources.
I propose that we start using terraform for managing our infrastructure.
This will open the door to actually start automating away most of the work that currently is just toil, and will also open the door to start building things on demand without the pain of dealing with a web environment that resists automation.
We could start with something really simple and small: importing the development fleet from digital ocean to be fully aware of it and remove out creating these droplets manually.
Steps Remaining:
- Dev
-
Resolve Consul issues with Terraform state -
True up environment view again with Terraform declarations -
Publish docs to explain how we're managing environments with Terraform MR's -
Convert all developer accounts in Digital Ocean to Read Only
-
- Staging
- Tracking in gitlab-com/infrastructure#1040
- Production
- Tracking in gitlab-com/infrastructure#1039