Updates security template to reflect new process

What does this MR do?

The security template was updated to reflect the new way of processing security merge requests:

• Approximately one week before the due date, security merge requests targeting master starts being merged
• During the Security Release, backports and security merge requests associated to other satellite projects are merged

Related to gitlab-com/gl-infra/delivery#1122 (closed)

Examples

Regular security release

Security patch release: 13.4.1, 13.3.7, 13.2.10

First steps

• Disable Omnibus nightly builds by setting the schedules to inactive: https://dev.gitlab.org/gitlab/omnibus-gitlab/pipeline_schedules.
• Ensure that staging, canary, and production are all running the same version from the same auto-deploy branch.
• Ensure latest auto-deploy branches are synced across Canonical, Security, and Build:

  # In Slack
  /chatops run mirror status

• Modify the dates below to accurate reflect the plan of action

Early-merge phase

Up until the 27th, or one day before the Security Release due date

• Verify if there are security issues ready to be processed

  # In Slack
  /chatops run release merge --dry-run --security --master

• If there security issues, proceed to merge the merge requests targeting master

  # In Slack
  /chatops run release merge --security --master

On the 27th (one day before due date)

If this date is on a weekend, do this work on the next working day.

• Notify AppSec Engineers that the Security Release has started. Link to the blog post on dev: https://dev.gitlab.org/gitlab/www-gitlab-com

• Merge security merge requests targeting master

# In Slack:
/chatops run release merge --security --master

• Merge backports and any other merge request pending:

# In Slack:
/chatops run release merge --security

• If any merge requests could not be merged, investigate what needs to be done to resolve the issues. Do not proceed unless it has been determined safe to do so.

On the Due Date

Packaging

• Ensure tests are green in CE and green in EE

  # In Slack:
  /chatops run release status --security

• Tag the 13.4.1 security release, and wait for the pipeline to finish: /chatops run release tag --security 13.4.1

• Tag the 13.3.7 security release, and wait for the pipeline to finish: /chatops run release tag --security 13.3.7

• Tag the 13.2.10 security release, and wait for the pipeline to finish: /chatops run release tag --security 13.2.10

Waiting between pipelines is necessary as they may othewise fail to concurrently push changes to the same project/branch.

• Check that EE and CE packages are built:
  • 13.4.1: EE packages and CE packages
  • 13.3.7: EE packages and CE packages
  • 13.2.10: EE packages and CE packages

Deploy

• Verify that release.gitlab.net is running the latest patch version
  • Check in Slack #announcements channel
  • Go to https://release.gitlab.net/help

Release

• Publish the packages via ChatOps:

   # In Slack:
    /chatops run publish 13.4.1
    /chatops run publish 13.3.7
    /chatops run publish 13.2.10
  

• Notify AppSec counterparts they can submit and merge the blog post to https://gitlab.com/gitlab-com/www-gitlab-com/

• Verify that EE packages appear on packages.gitlab.com: EE (should contain 14 packages)

• Verify that CE packages appear on packages.gitlab.com: CE (should contain 13 packages)

• Verify that Docker images appear on hub.docker.com: EE / CE

• Create the versions:

  • 13.4.1 version on version.gitlab.com. Be sure to mark it as a security release.
  • 13.3.7 version on version.gitlab.com. Be sure to mark it as a security release.
  • 13.2.10 version on version.gitlab.com. Be sure to mark it as a security release.

• In the #content-updates channel, share a link to the blog post.

Final steps

• Sync master and auto-deploy branches for GitLab, GitLab Foss, Omnibus GitLab and Gitaly, via ChatOps:

  # In Slack
  /chatops run release sync_remotes --security

• Verify all remotes are synced:

  # In Slack
  /chatops run mirror status

  If conflicts are found, manual intervention will be needed to sync the repositories.

• Close the security implementation issues

  # In Slack
  /chatops run release close_issues --security

• Enable Omnibus nightly builds by setting the schedules to active https://dev.gitlab.org/gitlab/omnibus-gitlab/pipeline_schedules

Critical security release

Security patch release: 13.4.1, 13.3.7, 13.2.10

First steps

• Disable Omnibus nightly builds by setting the schedules to inactive: https://dev.gitlab.org/gitlab/omnibus-gitlab/pipeline_schedules.
• Ensure that staging, canary, and production are all running the same version from the same auto-deploy branch.
• Ensure latest auto-deploy branches are synced across Canonical, Security, and Build:

  # In Slack
  /chatops run mirror status

One day before the due date

• Notify AppSec Engineers that the Security Release has started. Link to the blog post on dev: https://dev.gitlab.org/gitlab/www-gitlab-com
• Merge critical security merge requests using the UI.
  • Enable "Squash commits" option when merging.
• Cherry-pick the security fixes into the auto-deploy branch that is running on production.
• Wait for the tests on the auto-deploy branch to pass. This ensures that when we tag, we tag the security commits; not older commits.
• Deploy all the fixes to production.

On the Due Date

Packaging

• Ensure tests are green in CE and green in EE

  # In Slack:
  /chatops run release status --security

• Tag the 13.4.1 security release, and wait for the pipeline to finish: /chatops run release tag --security 13.4.1

• Tag the 13.3.7 security release, and wait for the pipeline to finish: /chatops run release tag --security 13.3.7

• Tag the 13.2.10 security release, and wait for the pipeline to finish: /chatops run release tag --security 13.2.10

Waiting between pipelines is necessary as they may othewise fail to concurrently push changes to the same project/branch.

• Check that EE and CE packages are built:
  • 13.4.1: EE packages and CE packages
  • 13.3.7: EE packages and CE packages
  • 13.2.10: EE packages and CE packages

Deploy

• Verify that release.gitlab.net is running the latest patch version
  • Check in Slack #announcements channel
  • Go to https://release.gitlab.net/help

Release

• Publish the packages via ChatOps:

   # In Slack:
    /chatops run publish 13.4.1
    /chatops run publish 13.3.7
    /chatops run publish 13.2.10
  

• Notify AppSec counterparts they can submit and merge the blog post to https://gitlab.com/gitlab-com/www-gitlab-com/

• Verify that EE packages appear on packages.gitlab.com: EE (should contain 14 packages)

• Verify that CE packages appear on packages.gitlab.com: CE (should contain 13 packages)

• Verify that Docker images appear on hub.docker.com: EE / CE

• Create the versions:

  • 13.4.1 version on version.gitlab.com. Be sure to mark it as a security release.
  • 13.3.7 version on version.gitlab.com. Be sure to mark it as a security release.
  • 13.2.10 version on version.gitlab.com. Be sure to mark it as a security release.

• In the #content-updates channel, share a link to the blog post.

Final steps

• Sync master and auto-deploy branches for GitLab, GitLab Foss, Omnibus GitLab and Gitaly, via ChatOps:

  # In Slack
  /chatops run release sync_remotes --security

• Verify all remotes are synced:

  # In Slack
  /chatops run mirror status

  If conflicts are found, manual intervention will be needed to sync the repositories.

• Close the security implementation issues

  # In Slack
  /chatops run release close_issues --security

• Enable Omnibus nightly builds by setting the schedules to active https://dev.gitlab.org/gitlab/omnibus-gitlab/pipeline_schedules