Monthly release: Test the release candidate automation on 18.2
Context
gitlab-com/gl-infra/software-delivery&1 (closed) proposes to move the active stable branch generation one week earlier in the release schedule to allow for thorough testing and timely backports. This will increase the quality, safety, and reliability of the monthly release while enabling a better developer experience and confidence.
The new workflow will be:
- 2nd Wednesday of the month: RC and Stable branch creation (after the patch release is out)
- Up to 3rd Tuesday of the month: Bug fixes are backported to the ongoing stable branch (see
- 3rd Tuesday of the month (Late EMEA or early AMER): Final RC is created
- 3rd Wednesday of the month Tag the final version
- 3rd Thursday of the month Release
Automation has been implemented to tag the initial and final release candidates
Test the release candidate automation on 18.2
18.2 is scheduled to be released next week. Lets use this opportunity to test the release candidate automation
Requirements
-
Update the monthly release pipeline to include the latest changes #21327 (closed) -
Update the monthly release issue gitlab-org/release/tasks#19933 (closed). It should still include the old steps in case the automation fails -
Enable the feature flag in ops: early_branch_creationhttps://ops.gitlab.net/gitlab-org/release/tools/-/feature_flags -
Notify release managers about the test
Initial release candidate (to be performed on 2025-07-15)
Job execution
-
Start the monthly_release_initial_rc:startjob -
Ensure the monthly_release_initial_rc:tagis executed after thestartjob -
After the tagjob completed:-
Review the verify_cejob is automatically triggered -
Review the update_branch_permissionsjob is automatically triggered -
Review the release_environmentjob is automatically triggered
-
-
90 minutes, after the tagjob completed:-
Review the verify_rc_pipelineis automatically triggered -
Review the verify_pre_deploymentis automatically triggered
-
monthly_release_initial_rc:start job
-
Ensure a notification is sent to the #f_upcoming_releasechannel
monthly_release_initial_rc:tag
-
Ensure the monthly_release_initial_rc:tagis successfully executed https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306140 -
Review the 18-2-stable-eebranch is created in GitLab repos-
Omnibus -
CNG -
Gitaly -
KAS -
GitLab Pages -
GitLab -
GitLab FOSS - [-] Helm => No RC's are created for Helm
-
-
Review the 18.2.0-rc42tag is created on the GitLab repos-
Omnibus (2 tags should be available) -
CNG (4 tags should be available) -
Gitaly -
KAS -
GitLab Pages -
GitLab -
GitLab FOSS - [-] Helm => No RC's are created for Helm
-
monthly_release_initial_rc:verify_ce
-
Ensure the monthly_release_initial_rc:verify_cejob is successfully executed: https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306141 -
Verify that a Slack notification was sent -
Verify that the CE stable branch contains the right commits - There should be at least two commits: the last commit from the previous stable branch (usually a version update), and the sync commit created by the merge train.
- The sync commit will have the message "Add latest changes from gitlab-org/gitlab@18-2-stable-ee"
monthly_release_initial_rc:verify_rc_pipeline
-
Ensure the monthly_release_initial_rc:verify_rc_pipelineis successfully executed https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19323539 -
Verify that a Slack notification was sent -
Verify that pipelines are green /chatops run release status 18.2.0-rc42
monthly_release_initial_rc:verify_pre_deployment
-
Ensure the monthly_release_initial_rc:verify_pre_deploymentis successfully executed https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306143 -
Verify that a Slack notification was sent -
Verify that the RC has been deployed to the pre environment - Deployment to pre will start automatically. It can take 2 hours to start once the RC is tagged. A notification will be sent to the
#announcementschannel in Slack when it starts.
- Deployment to pre will start automatically. It can take 2 hours to start once the RC is tagged. A notification will be sent to the
monthly_release_initial_rc:update_branch_permissions
-
Ensure the monthly_release_initial_rc:update_branch_permissionsis successfully executed https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306144 -
Verify that a Slack notification was sent -
Review the permissions for the 18-2-stable-eebranch are granted to maintainers and release managers
monthly_release_initial_rc:release_environment
-
Ensure the monthly_release_initial_rc:release_environmentis successfully executed https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306145 -
Verify that a Slack notification was sent -
Review the release environment is automatically triggered #21309 (closed) => https://gitlab.com/gitlab-org/security/gitlab/-/commit/89165ab9e37e08bae73a47613499437720b568c4/pipelines?ref=18-2-stable-ee
Clean-up
This is not needed if the next release candidate can be created
Final release candidate (to be performed on 2025-07-15)
Job execution
-
Start the monthly_release:final_rc:startjob https://ops.gitlab.net/gitlab-org/release/tools/-/pipelines/4727877 -
Ensure the monthly_release_final_rc:verify_stable_branchis executed after thestartjob -
Ensure the monthly_release_final_rc:update_branch_permissionis executed after thestartjob -
Ensure the monthly_release_final_rc:tagis executed after theverify_stable_branchandupdate_branch_permissionsjobs -
After the tagjob completed, review themonthly_release_final_rc:verify_cejob is automatically triggered -
90 minutes, after the tagjob is completed:-
Review the verify_rc_pipelineis automatically triggered -
Review the verify_pre_deploymentis automatically triggered
-
monthly_release:final_rc:start job
-
Ensure a notification is sent to the #f_upcoming_releasechannel
monthly_release_final_rc:verify_stable_branch
-
Ensure the monthly_release_final_rc:verify_stable_branchjob executes successfully https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19324766 -
Review the stable branch for GitLab projects and ensure it matches the job outcome
monthly_release_final_rc:update_branch_permission
-
Ensure the monthly_release_final_rc:update_branch_permissionjob executes successfully https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306148 -
Review the permissions for the 18-2-stable-eebranch are restricted to release managers
monthly_release_final_rc:tag
-
Ensure the monthly_release_final_rc:tagjob executes sucessfully https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306149 -
Confirm the RC43was automatically assigned -
Review the 18.2.0-rc43tag is created on the GitLab repos-
Omnibus (2 tags should be available) -
CNG (4 tags should be available) -
Gitaly -
KAS -
GitLab Pages -
GitLab -
GitLab FOSS
-
monthly_release_final_rc:verify_ce
- [-] Ensure the
monthly_release_final_rc:verify_cejob executes successfully https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/19306150 - [-] Verify that the CE stable branch contains the right commits
- There should be at least two commits: the last commit from the previous stable branch (usually a version update), and the sync commit created by the merge train.
- The sync commit will have the message "Add latest changes from gitlab-org/gitlab@18-2-stable-ee"
monthly_release_final_rc:verify_rc_pipeline
-
Ensure the monthly_release_initial_rc:verify_rc_pipelineis successfully executed -
Verify that pipelines are green /chatops run release status 18.2.0-rc43
monthly_release_final_rc:verify_pre_deployment
-
Ensure the monthly_release_final_rc:verify_pre_deploymentis successfully executed -
Verify that the RC has been deployed to the pre environment - Deployment to pre will start automatically. It can take 2 hours to start once the RC is tagged. A notification will be sent to the
#announcementschannel in Slack when it starts.
- Deployment to pre will start automatically. It can take 2 hours to start once the RC is tagged. A notification will be sent to the
Clean-up
Follow-ups
- Early branch: Minor adjustments of the monthly ... (#21342 - closed)
- Early branch: Deployer user not available on Sl... (#21344 - closed)
- Early branch: Adjusts stable branch permissions... (#21345 - closed)
- Follow-up: Validate if release environment is c... (#21309 - closed)
- Early branch: Adjust the `verify_pre_deployment... (#21347 - closed)
- Early branch: Adjust the `verify_rc_pipeline` job (#21348 - closed)
- Early branch: Adjust the verify stable branch o... (#21349 - closed)
- Early branch: Adjust the `verify_ce` job on the... (#21350 - closed)
Table view of the follow-up issues
display: table
title: Follow-up issues for earlier stable branch creation
description: This view lists my open issues
fields: title, state, assignee, updated
limit: 20
query: project = "gitlab-com/gl-infra/delivery" and id in (21342, 21344, 21345, 21309, 21347, 21348, 21349, 21350)
Conclusion
- Release candidates for 18.1 and 18.2 were successfully created. Details: initial and final
- Follow-up issues have been opened to address issues encountered with the automation
- Another test will be performed on 18.3 to gather release manager feedback #21356 (closed)
Edited by Siddharth Kannan