Skip to content

Removes time restrictions on RC tagging

John Skarbek requested to merge jts/remove-time-restrictions into master
  • Back in the day, we used to run RC's so timing may have been important when it came to validating packages were running in production
  • Now with auto-deploy, our RC is now very behind production
  • Once an RC is created, why wait for specific times for which we can start tagging?
  • With this change, we keep the date for which things happen:
    • This allows us plenty of time to validate that the RC is built, and that the tasks are checked off in the QA task list
    • While we don't watch the task list ourselves, waiting this extra day should provide us with ample time to be notified by any member of Engineering if something is wrong with the RC
  • So after a day has passed, we can proceed with tagging, while validating individual steps
    • Such as deployment to the release environment
    • and that we've passed QA
Example Monthly Procedure

Release 42.42

First steps

  • Join the slack channel #f_upcoming_release
  • Change the channel topic with /topic Issue: <link_to_this_issue>
  • Invite all current release managers
  • Modify the dates below to accurately reflect the plan of action

Monitor

  • Monitor QA issues for each deploy to gstg
  • Ensure any deploys that do not make it to canary are investigated
  • Try to push any successful deploy to canary into production after some time has passed, preferably after all QA tasks are accounted and checked
  • Monitor for any blockers to production and determine if canary should be disabled

17th

If this date is on a weekend, do this work on the next working day

  • Find the latest sha that made it into production successfully: sha
  • Notify Engineering Managers that this is the sha that is guaranteed to be released on the 22nd, in #releases: 
    :mega: This is the most recent commit running on GitLab.com and this is guaranteed to be released on the 22nd.
https://gitlab.com/gitlab-org/gitlab/commits/<SHA>
Please see the following documentation on what this means:
  * `https://about.gitlab.com/handbook/engineering/releases/#how-can-i-determine-if-my-merge-request-will-make-it-into-the-monthly-release`
  * `https://about.gitlab.com/handbook/engineering/releases/#when-do-i-need-to-have-my-mr-merged-in-order-for-it-to-be-included-into-the-monthly-release`
  • Link the above message to additional channels: #development, #backend, and #frontend

18th

If this date is on a weekend, do this work on the last Friday before the 18th.

  • Proceed to cut a new auto-deploy branch 
    # In Slack:
/chatops run auto_deploy prepare
  • Ensure this build makes it through into production
  • Grab the sha from this new auto-deploy branch and notify Engineering Managers that this is the candidate sha for the release, in #releases: 
    :mega: This is the _candidate_ commit to be released on the 22nd.
https://gitlab.com/gitlab-org/gitlab/commits/<SHA>
Please see the following documentation on what this means:
  * `https://about.gitlab.com/handbook/engineering/releases/#how-can-i-determine-if-my-merge-request-will-make-it-into-the-monthly-release`
  * `https://about.gitlab.com/handbook/engineering/releases/#when-do-i-need-to-have-my-mr-merged-in-order-for-it-to-be-included-into-the-monthly-release`
  • Link the above message to additional channels: #development, #backend, and #frontend

20th: two working days before the release

If this date is on the weekend, do this work on the last Friday before the 20th.

  • Determine what the last green auto deploy branch is and add it here: BRANCH

  • Create the stable branches using chatops by running the following in Slack: /chatops run release stable_branch 42.42.0

  • Verify that the CE stable branch contains the right commits

    • There should be at least two commits: the last commit from the previous stable branch (usually a version update), and the sync commit created by the merge train.
    • The sync commit will have the message "Add latest changes from gitlab-org/gitlab@42-42-stable-ee"

  • Create a RC version to ensure that the final version builds correctly

    # In Slack:
/chatops run release tag 42.42.0-rc42

  • Notify Engineering Managers that final candidate has been created in #releases:

    :mega: Barring any show-stopping issues, this is the final commit to be released on the 22nd.
https://gitlab.com/gitlab-org/gitlab/-/commits/42-42-stable-ee

21st: one day before the release

Instructions for manual deploy 
    ```sh
    # In Slack:
    /chatops run deploy 42.42.0-ee.0 --release
    ```
  • Validate 42.42.0 has been passed automated QA

Past this point, no new code can be added to the release that was not included in the final RC.

22nd: release day

Final release is tagged, so any changes will have to initiate a patch release.

  • At 13:00 UTC, post an update about the package building status in #f_upcoming_release
  • At 13:30 UTC:
    • ⚠ Make sure that neither packages nor the blog post get published earlier than 13:30UTC without approval by the messaging lead of the release post. Mind that you don't need their approval if you're on time ⚠
    • Publish the packages via ChatOps: 
      # In Slack:
/chatops run publish 42.42.0
    • If anything goes wrong and the release is delayed, ping the release post manager on Slack to make them aware of the issue. Cross-post the slack message to the #marketing channel to notify them too
  • At 14:10 UTC:
    • Verify that packages appear on packages.gitlab.com: EE / CE
    • Verify that Docker images appear on hub.docker.com: EE / CE
    • Create the 42.42.0 version on version.gitlab.com
    • Post an update about the status in #f_upcoming_release
    • Once all packages are available publicly and GitLab.com is up and running on the release version, ping the release post manager on Slack (#release-post channel) to give them a go to merge the release post at ~14:20 UTC, so that it will be live at 15:00 UTC
  • Open MR to Bump VERSION file to the next version to be released
Edited by John Skarbek

Merge request reports