Skip to content

Revamp the deployment procedure using Terraform

Building on top of using terraform to provision new hosts bootstrapped with chef we need to change our deployment procedure such as:

  1. We install the new version in the deployment host.
  2. We run migrations in this host.
  3. In parallel, we spin up a new fleet with the new version of the package.
  4. We validate that there are no remaining migrations.
  5. We direct the traffic in the front end load balancers to the new fleet.
  6. We wait until all the connections to the old fleet are drained.
  7. We stop mailing and sidekiq in the old fleet.
  8. We enable sidekiq and mailing in the new fleet.
  9. We execute post-deployment migrations.
  10. We destroy the old fleet.

Multiple advantages here:

  1. We have simple and immediate rollback plan.
  2. We will not be attacked by unpackaging or apt issues anymore.
  3. The switchover happens at LB level, so it's immediate
  4. Any ongoing connection will just continue.
  5. This could open the door to have canary deployment with our current setup (if the database model is the same)

Disadvantages:

  1. We need to scale connection to keep 2 fleets running at any time.
  2. We will be spending a bit more money while keeping 2 front end fleets at the same time.

cc/ @gl-infra

Edited by Craig Miskell