## Summary dev.gitlab.org is a [SPOF](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/1291) and currently sits in legacy Azure. Due to this we have limited visibility into metrics, alerting, and maintenance requires access to the Azure cloud account. As the dev runners are living in GCP, while the dev-registry buckets are living in AWS and assets are uploaded to GCP we see huge delays and timeouts in our release and deployment pipelines as we are mostly limited by network bandwidth between GCP, Azure and AWS. This epic tracks the work to do a minimal "lift-and-shift" of dev.gitlab.org to GCP, we will maintain a single VM, and continue to use community edition with a nightly build. We may want to consider using maintenance mode https://gitlab.com/groups/gitlab-org/-/epics/2149, which will ease the transition, especially if this allows us to continue to pull from the registry which has many dependencies in both release and build pipelines. ## Milestones Main tasks need to be serialized, but within each of them we can parallelize some of the sub-tasks depending on resources. 1. [x] Planning and infra setup 1. [x] Identify dependencies https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/1456 1. [x] [Create new GCP project](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2196) with terraform (1d) * project, service-accounts, GKMS, ... 1. [x] [Create necessary infrastructure](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2197) with terraform (1d) * dev instance, buckets, bastion, prometheus, ... 1. [x] [Create new env in chef, create chef roles](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2198) (1d) 1. [x] [Check gitlab setup](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2204) on new node 1. [x] [Create migration plan and CR](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2205) (4h) 1. [x] prepare MR for increasing min replicas 1. [x] [Sync data and test](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2199) 1. [x] Initial sync of data (1d) 1. [x] test an incremental sync to get time estimates for the switch-over (2h) 1. [x] Test and [fix remaining issues](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2204) (1d) 1. [x] execute [Migration](https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6236) (1h) 1. [x] Cleanup (2d) 1. [x] [Setup DB backups](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2202) 1. [x] [Setup DB restore tests](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2215) 1. [x] [Decommission old azure infra](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2207), S3 bucket, update runbooks For any issue that are to be associated with this epic, use the following: ## DRI @hphilipps ## Issue Template ``` /epic &389 /label ~"team::Delivery" ~"Delivery::P2" ~"workflow-infra::Ready" ~"release tooling" ``` ## Status 2022-02-08 We finished the remaining tasks for this epic (adding [backup verification tests](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2215) and [decommissioning old instance](https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2207)). With that this epic is complete and a separate epic will track the tasks needed to avoid Dev being a SPOF: https://gitlab.com/groups/gitlab-com/gl-infra/-/epics/669