## Summary This issue is to roll out [the feature](https://gitlab.com/gitlab-org/gitlab/-/work_items/410143) on production, that is currently behind the `ci_lazy_predefined_variables` feature flag. ## Owners - Most appropriate Slack channel to reach out to: `#g_pipeline-authoring` - Best individual to reach out to: @lauraXD ## Expectations ### What are we expecting to happen? ## Rollout Steps Note: Please make sure to run the chatops commands in the Slack channel that gets impacted by the command. ### Rollout on non-production environments * [x] Verify the MR with the feature flag is merged to `master` and has been deployed to non-production environments with `/chatops gitlab run auto_deploy status <merge-commit-of-your-feature>` * [x] Enable the feature globally on non-production environments with `/chatops gitlab run feature set ci_lazy_predefined_variables true --dev --pre --staging --staging-ref` ### Specific rollout on production For visibility, all `/chatops` commands that target production must be executed in the [`#production` Slack channel](https://gitlab.slack.com/archives/C101F3796) and cross-posted (with the command results) to the responsible team's Slack channel. - [ ] Depending on the [type of actor](https://docs.gitlab.com/development/feature_flags/#feature-actors) you are using, pick one of these options: - [x] `/chatops gitlab run feature set --project=gitlab-org/lauraXD ci_lazy_mr_approval_variables true` - [ ] `/chatops gitlab run feature set --project=gitlab-org/gitlab,gitlab-org/gitlab-foss,gitlab-com/www-gitlab-com ci_lazy_mr_approval_variables true` - [ ] Verify that the feature works for the specific actors. ### Preparation before global rollout - [x] Set a milestone to this rollout issue to signal for enabling and removing the feature flag when it is stable. - [x] Ensure that you or a representative in development can be available for at least 2 hours after feature flag updates in production. If a different developer will be covering, or an exception is needed, please inform the oncall SRE by using the `@sre-oncall` Slack alias. - [ ] [Incrementally roll out](https://docs.gitlab.com/development/feature_flags/controls/#process) the feature on production. * [ ] `/chatops gitlab run feature set ci_lazy_predefined_variables 30 --actors`. * [ ] `chatops gitlab run feature set ci_lazy_predefined_variables 70 --actors`. * [ ] `chatops gitlab run feature set ci_lazy_predefined_variables true` - Between every step wait for at least 15 minutes and monitor the appropriate graphs on https://dashboards.gitlab.net. - [ ] After the feature has been 100% enabled, wait for [at least one day before releasing the feature](#release-the-feature). ### Release the feature You can either [create a follow-up issue for Feature Flag Cleanup](https://gitlab.com/gitlab-org/gitlab/-/issues/new?issuable_template=Feature%20Flag%20Cleanup) or use the checklist below in this same issue. <!--The checklist here is to help stakeholders keep track of the feature flag status--> - [ ] Create a merge request to remove the `ci_lazy_predefined_variables` feature flag. Ask for review/approval/merge as usual. - [ ] Ensure that the cleanup MR has been included in the release package. If the merge request was deployed before [the monthly release was tagged](https://about.gitlab.com/handbook/engineering/releases/#self-managed-releases-1), the feature can be officially announced in a release blog post: `/chatops gitlab run release check <merge-request-url> <milestone>` - [ ] Once the cleanup MR has been deployed to production, clean up the feature flag from all environments by running these chatops command in `#production` channel: `/chatops gitlab run feature delete ci_lazy_predefined_variables --dev --pre --staging --staging-ref --production` ## Rollback Steps - [ ] This feature can be disabled on production by running the following Chatops command: ``` /chatops gitlab run feature set ci_lazy_predefined_variables false ``` - [ ] Disable the feature flag on non-production environments: ``` /chatops gitlab run feature set ci_lazy_predefined_variables false --dev --pre --staging --staging-ref ``` - [ ] Delete feature flag from all environments: ``` /chatops gitlab run feature delete ci_lazy_predefined_variables --dev --pre --staging --staging-ref --production ```