In https://gitlab.com/gitlab-org/release/framework/issues/161 we are looking to create daily deploy to staging.gitlab.com. With increased use of canary in issues https://gitlab.com/gitlab-org/release/framework/issues/140 and https://gitlab.com/gitlab-org/release/framework/issues/156, we are going to have a process for promoting to canary and production services This epic makes up all the tasks to get us to more frequent (daily) deployments to all Gitlab.com environments and implements [the design for daily deployments](https://about.gitlab.com/handbook/engineering/infrastructure/design/scheduled-daily-deployments/) ## Daily deployments to staging ### Project tracking * Use the [daily deployment to staging](https://gitlab.com/gitlab-org/release/framework/issues?label_name%5B%5D=daily_deployments%3Astaging) label or the [issue board](https://gitlab.com/gitlab-org/release/framework/boards/998366?&label_name[]=daily%20deployments) * All work is tagged with priority labels indicating the order that they need to be done ### Press release > Today the Delivery team is happy to announce that we have increased the number of deployments to staging from an approximate weekly cadence with selected changes after the monthly code freeze, to daily deployments that take changes directly from the master branch at a frequency of at least once a week. ### Overview In order to enable frequent deployments to staging we are moving to a branch based model for deployments where the latest commit on the gitlab-ee [auto-deploy branch](https://gitlab.com/gitlab-org/release/framework/issues/259) will be deployed as frequently as changes are committed to it. New [picking automation](https://gitlab.com/gitlab-org/release/framework/issues/252) will be implemented to allow developers to label MRs that have been merged to master to be picked up for the most current auto-deploy branch for a MAJOR.MINOR release, example: `~Pick into 11.10` Because releases to staging are not tagged, we will rely on version queries to know exactly what is run on staging, [this will be implemented in chatops](https://gitlab.com/gitlab-org/release/framework/issues/258). This will allow us to query version SHAs of components (gitaly, gitlab-ee, workhorse, etc) that are deployed to staging. We will use a modified [omnibus build pipeline](https://gitlab.com/gitlab-org/release/framework/issues/260) that uses what we have now for building omnibus packages from a branch (by setting the omnibus `VERSION` to the branch name), plus it will only build what is necessary for gitlab.com and trigger the staging deploy pipeline. ### Design This implements the design https://about.gitlab.com/handbook/engineering/infrastructure/design/scheduled-daily-deployments/ for how we intend to increase the frequency of deployments. ### How this is different than the current process? * More frequent deploys to staging * Instead of a stable branch created on the 7th via chatops, a periodic job will create an _auto-deploy_ branch from master as frequently as we like, once a week to start * Instead of tagging to do a deploy, changes made to the _auto-deploy_ branch will automatically be deployed to staging * The responsibility for resolving conflicts for picking changes will be driven by the developer authoring the change, if there is a conflict, a new MR will be created and assigned to the author that targets the _auto-deploy_ branch. See https://gitlab.com/gitlab-org/release/framework/issues/252 #### ~ Once a week New branches are created in `gitlab-ee`, `gitlab-ce` and `omnibus-gitlab`, the naming of this branch will be consistent across all repositories `$MAJOR-$MINOR-auto-deploy-$CI_PIPELINE_IID-ee` where VERSION the latest active milestone for gitlab-ce. `$CI_PIPELINE_IID` is the pipeline ID of the [release-tools](https://gitlab.com/gitlab-org/release-tools) job that is run to create the jobs #### ~ Every day (or more frequently) Changes will be picked, gitlab-ce will be merged into gitlab-ee and pushed to the corresponding auto-deploy branch on dev.gitlab.org #### Whenever a change is pushed to an auto-deploy branch After tests pass, an omnibus pipeline will run which will build the package we need for gitlab.com and trigger a deployment pipeline to staging