Incremental rollouts (manual)
Description
When you have a new version of your app to deploy, you may want to use incremental rollouts to replace just a few pods with the latest code. This will allow you to check how it is going, and then increase up to 100%
We should provide such flow in Auto DevOps, so users can leverage incremental rollouts easily.
Owners
BE: @mayra-cabrera
Proposal
Allow users to specify if they want to use incremental rollouts for their production
deployments via the $INCREMENTAL_ROLLOUTS_ENABLED
variable.
If yes, instead of the standard production
job different jobs will be created.
rollout 10%
rollout 25%
rollout 50%
rollout 100%
Job definitions are something similar to this:
.production: &production
stage: production
script:
- echo "Standard Deploy"
- deploy
environment:
name: production
url: http://$CI_PROJECT_PATH_SLUG.example.com
only:
refs:
- master
production:
<<: *production
except:
variables:
- $INCREMENTAL_ROLLOUT_ENABLED
rollout 10%:
<<: *production
script:
- echo "Incremental Deploy 10%"
- deploy rollout 10
only:
variables:
- $INCREMENTAL_ROLLOUT_ENABLED
rollout 25%:
<<: *production
script:
- echo "Incremental Deploy 25%"
- deploy rollout 25
when: manual
only:
variables:
- $INCREMENTAL_ROLLOUT_ENABLED
rollout 50%:
<<: *production
script:
- echo "Incremental Deploy 50%"
- deploy rollout 50
when: manual
only:
variables:
- $INCREMENTAL_ROLLOUT_ENABLED
rollout 100%:
<<: *production
script:
- echo "Incremental Deploy 100%"
- deploy rollout 100
when: manual
only:
variables:
- $INCREMENTAL_ROLLOUT_ENABLED
The first one (10%) will be auto, the other three will be when: manual
. The last three will automatically appear in the dropdown for the production
environment
When an incremental rollout job is done, it should:
- scale the new version to the required percentage
- scale the old version to reach the total number of pods required by
$REPLICAS
Possible edge cases
- when 100% is reached, scaling down the rollout should be avoided
- since we can have
staging
, bothproduction
androllout 10%
should be duplicated to allow auto/manual flow - redeploy button should consider incremental rollouts when choosing the job to run