Use gcs as remote backend for deployment projects
As a follow up to #230 (closed), having deployment projects use gcs instead of gitlab to manage the terraform state would allow both .com and ops environments to access the state file.
This gives us the following benefits
- smoother migration path (we can toggle the CI yml to cutover
reconciler deployfrom .com to ops) - .com pipeline can run plan, allowing users to iterate without having ops access. The .com pipeline can work with a readonly roleset too.
Implementation ideas
- Configure using a
.tfbackendfile (https://developer.hashicorp.com/terraform/language/settings/backends/configuration#file) and switch betweenhttp.tfbackendandgcs.tfbackendso that other runway deployment projects can continue usinghttpbackend. - Create bucket in provisioner within the runway gcp project in which the service is deployed.
Some complexities to deal with
- we cannot use
gitlab-terraformsince that is meant to target gitlab as a http remote backend. We will need to switch betweengitlab-terraformandterraformdepending on http/gcs backend. -
.tfbackendfiles cannot configure thetypeof backend (e.g.httporgcs), only the configuration values IIUC.runwayctlwill have to do so file manipulation to write the correctbackend.tffile during init. I fear this could break the destroy workflow since we have abootstrap.tf
When using partial configuration, Terraform requires at a minimum that an empty backend configuration is specified in one of the root Terraform configuration files, to specify the backend type
Note: this is considered a good-to-have and should be prioritised lower than other reliability improvement issues.
/cc @gsgl thanks for the great idea, please update as required
update: dropping this issue from the &1348 epic since it is not a hard requirement for decoupling deployment from .com
Edited by Sylvester Chin