Skip to content

Allow only forward incremental deployments

The problem

As part of introducing Incremental Rollouts we hit a bunch of problems (described in https://gitlab.com/gitlab-org/gitlab-ee/issues/1660#note_100311728):

  1. If a new pipeline is pushed, the already scheduled delayed jobs gonna be executed, even though that they do incremental rollout of older pipelines,
  2. Currently, this can be avoided only by a user doing a specially crafted script to check that,
  3. This is a problem as we don't have a concept of fowrad deployments,
  4. Stop environment can be executed for past deployment on environment, where we should allow only the latest one.

The solution

However, with the introduction of Make deployments stateful we keep track of all future deployments, including their statuses. This makes it possible for us to solve the problem of concurrent deploys:

  1. When we try to run deployment, we verify if our deployment is latest one,
  2. We allow the deployment to run, thus job to be pending/running when the last success/failed deployment sha is equal to ours, or our deployment is never than latest success/failed one,
  3. We apply this rule to every CI job doing the action on environment.

The example

class Build
  enum :failure_reason {
    out_of_date_deployment_action: ...
  }

  def is_latest_deployment_on_environment?
    environment = persisted_environment
    return unless environment

    last_deployment = environment.last_deployment
    return true unless last_deployment

    if starts_deployment?
      # if our deployment is later than last successful deployment
      # it should fullfil the rule of being a forward deployment
      # if this is incremental deployment it will always be later in the pipeline
      deployment.id >= last_deployment.id
    elsif stops_deployment?
      # if we want to stop, we require that stop action is for the same SHA
      last_deployment.sha == sha
    end
  end
end

UX proposal

A new setting will be added under CI/CD settings -> General Pipelines called Drop older active deployment as described below

https://gitlab.com/gitlab-org/gitlab/uploads/396e472499c22dfb2cc45efb478dd6eb/Screenshot_from_2020-02-19_15-31-57.png

Edited by Orit Golowinski