Rollout plan for Security release changes
Overview
This issue describes the end-goal workflows for those effected by &1061 (closed) and then describes the rollout plan and timeline regarding these changes.
Affected Positions
- Engineers
- Release Managers
Process of getting an issue into a release (engineer's point of view)
- An engineer opens a security implementation issue to complete a security fix
- The implementation issue automatically gets assigned the security-notifications label
- The engineer completes the MR targeting the default branch and gets it reviewed and approved
- The engineer opens 2 backports, but cannot open the 3rd because the stable branch does not yet exist for the active version.
- The stable branch is created for the active version and the engineer gets pinged on the implementation issue that the branch exists and they can create the last backport.
- The engineer completes the last backport
- The engineer adds the security-target label because they believe the issue to be ready for release.
- Release-tools processes the issue and links it to the security release tracking issue, notifying the engineer that no further action is needed.
- Alternatively, release-tools processes the issue and rejects it, notifying the engineer of what needs to be fixed and removing the security-target label.
- The engineer then fixes things and reapplies the security-target issue until it is successfully linked.
- The issue is released with the next security release.
Process of releasing an issue (release manager's point of view)
- The previous release managers complete a release, creating a new stable branch in the process.
- The release manager sets a due date for the security release.
- Release-tools is constantly processing and linking/unlinking issues from the security release tracking issue as they are ready.
- Release managers are pinged if there are issues not handled by the processor (managed versioning projects, for example).
- One or two days before the security release, the processor is deactivated by the release manager, preventing any new issues. (This will eventually be automated as the early merge phase gets shorter and more consistent).
- The release manager runs the default branch merge command:
/chatops run release merge --security --default-branch
, merging the default MRs for all issues linked to the security release tracking issue. They are all deployed in the same deployment. - One day before the security release, release managers verify everything has made it to production and then merges the backports. The rest of the security release follows the existing workflow.
- The security release is completed.
- The processor is re-activated.
Rollout Timeline and Tasks
2023-09-27
-
Announce dates of importance with development - #19686 (closed)
2023-10-10
-
Repeat dates development announcement letting folks know we are officially changing over to the new process this week. -
Enable processor scheduled pipeline after the stable branch has been created. -
Update all linked security implementation issues with security-notifications and update the steps, pinging the assignee to let them know. -
Update all open security implementation issues with security-notifications and update the steps, pinging the assignee to let them know. -
Unlink a few of the issues after discussing and getting an OK from the authors (we don't want to cause confusion or worry by unlinking everything).
Days leading up to the next scheduled security release
-
Check in every few days to see if anyone is still manually linking or following the old template and ping them reminding them of the new process. - Tracking issue: https://gitlab.com/gitlab-org/gitlab/-/issues/426612
- Issues with security-target label: https://gitlab.com/gitlab-org/security/gitlab/-/issues/?sort=created_date&state=opened&label_name%5B%5D=security-target&first_page_size=20
- Gitaly security issues: https://gitlab.com/gitlab-org/security/gitaly/-/issues
- Omnibus security issues: https://gitlab.com/gitlab-org/security/omnibus-gitlab/-/issues
- CNG security issues: https://gitlab.com/gitlab-org/security/charts/components/images/-/issues
2023-10-17
-
Share a E2E demo with Delivery during the adapt release process demo
2023-10-19
-
Repeat general announcement in slack to spread awareness of changes noting stable branch has been created. -
Enable automatic_picking_process
feature flag to modify the security release task issue (this just needs to be enabled before the new security release task issue is created, which is usually after the 22nd).
2023-10-25
-
Final development announcement after the 22nd release reminding everyone to create the last backport before applying security-target.
Around 2023-11-01 (will be adjusted based on security release date)
-
Request feedback after the security release is completed linking to the two feedback issues
References
Links
- Feedback issue for Development #19677 (closed)
- Feedback issue for Release managers #19676 (closed)
Slack channels for notifications
-
#engineering-fyi
(and supporting doc) #development
#backend
#backend_maintainers
#frontend
#frontend_maintainers
#gitlab-pages
#g_gitaly
#g_distribution
#sec_appsec
#releases
#eng-managers
-
#whats-happening-at-gitlab
(maybe only one announcement here)
Edited by Steve Abrams