Separate the creation of release environment from tagging

Mawreen Dela Cruzrequested to merge
mdc/separate-re-from-tagging into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Currently, a release environment is created when a monthly is tagged (part of the release:tag rake task). Due to that, when the release environment creation failed and we had to re-run it, we would need to retrigger the whole tagging job.

To save time and resources, this MR separates the release environment creation from tagging. This change will be under a feature flag early_branch_creation.

A new job monthly_release:initial_rc:release_environment is added to the initial_rc stage of the monthly release pipeline that is specific for this purpose.

Related issue: gitlab-com/gl-infra/delivery#21147 (closed)

Testing

  1. Test the change in the release:tag rake task to skip the creation of the release environment when skip_release_environment_creation feature flag is enabled (code change)
  • When early_branch_creation is enabled, creation of RE is skipped when tagging the monthly release
 
 bundle exec rake 'monthly:tag_day:tag'
2025-05-27 13:05:59.878336 W ReleaseTools::Feature -- Unleash::Client is disabled! Will only return default (or bootstrapped if available) results!
2025-05-27 13:05:59.878832 W ReleaseTools::Feature -- Unleash::Client is disabled! Metrics and MetricsReporter are also disabled!
2025-05-27 13:06:00.660541 I [dry-run] ReleaseTools::Tasks::Release::Tag -- Starting to tag 18.1.0
  • When early_branch_creation is disabled, RE creation is run as part of monthly release tagging
 
bundle exec rake 'monthly:tag_day:tag'
2025-05-27 13:13:00.323072 W ReleaseTools::Feature -- Unleash::Client is disabled! Will only return default (or bootstrapped if available) results!
2025-05-27 13:13:00.323474 W ReleaseTools::Feature -- Unleash::Client is disabled! Metrics and MetricsReporter are also disabled!
2025-05-27 13:13:01.107953 I [dry-run] ReleaseTools::Tasks::Release::Tag -- Starting to tag 18.1.0
2025-05-27 13:13:01.108104 I [dry-run] ReleaseTools::Tasks::ReleaseEnvironment::Environment -- Creating a release environment -- {version: "18.1.0", suffix: "security"}
2025-05-27 13:13:01.612286 D [dry-run] ReleaseTools::GitlabClient -- 200 "GET https://gitlab.com/api/v4/projects/gitlab-com%2Fgl-infra%2Frelease-environments/repository/tree" - 
2025-05-27 13:13:02.116049 D [dry-run] ReleaseTools::GitlabClient -- 404 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee" 34 
2025-05-27 13:13:04.071723 D [dry-run] ReleaseTools::GitlabClient -- 404 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee" 34 
rake aborted!
2025-05-27 13:13:05.835271 D [dry-run] ReleaseTools::GitlabClient -- 404 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee" 34 
KeyError: key not found: "CI_JOB_URL" (KeyError)
2025-05-27 13:13:05.836041 F [dry-run] Rake::Task -- Task failed -- Exception: Gitlab::Error::NotFound: Server responded with code 404, message: 404 Commit Not Found. Request URI: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee

2025-05-27 13:13:05.836427 F [dry-run] ReleaseTools::Tasks::Release::Tag -- Tagging the monthly release failed with error: Server responded with code 404, message: 404 Commit Not Found. Request URI: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee.
Review the error log and consider retrying this job.
 -- {error: #<Gitlab::Error::NotFound: Server responded with code 404, message: 404 Commit Not Found. Request URI: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee>}
2025-05-27 13:13:05.836614 I [dry-run] ReleaseTools::Slack::ReleaseJobEndNotifier -- Posting slack message -- {job_type: "Tag monthly release version 18.1.0", status: :failed}
  1. Running bundle exec rake 'monthly:initial_rc:release_environment' in dry-run mode
  • Output when early_branch_creation is enabled
 
bundle exec rake 'monthly:initial_rc:release_environment'
2025-05-27 13:17:02.859567 W ReleaseTools::Feature -- Unleash::Client is disabled! Will only return default (or bootstrapped if available) results!
2025-05-27 13:17:02.859951 W ReleaseTools::Feature -- Unleash::Client is disabled! Metrics and MetricsReporter are also disabled!
2025-05-27 13:17:03.302718 I [dry-run] ReleaseTools::Monthly::ReleaseCandidate::CreateReleaseEnvironment -- Creating release environment -- {version: "18.1"}
2025-05-27 13:17:03.302770 I [dry-run] ReleaseTools::Tasks::ReleaseEnvironment::Environment -- Creating a release environment -- {version: "18.1", suffix: "security"}
2025-05-27 13:17:03.809605 D [dry-run] ReleaseTools::GitlabClient -- 200 "GET https://gitlab.com/api/v4/projects/gitlab-com%2Fgl-infra%2Frelease-environments/repository/tree" - 
2025-05-27 13:17:04.772850 D [dry-run] ReleaseTools::GitlabClient -- 404 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee" 34 
2025-05-27 13:17:05.681114 D [dry-run] ReleaseTools::GitlabClient -- 404 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee" 34 
rake aborted!
2025-05-27 13:17:07.511519 D [dry-run] ReleaseTools::GitlabClient -- 404 "GET https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee" 34 
KeyError: key not found: "CI_JOB_URL" (KeyError)
2025-05-27 13:17:07.512082 E [dry-run] ReleaseTools::Monthly::ReleaseCandidate::CreateReleaseEnvironment -- Release environment creation failed -- {error: "Server responded with code 404, message: 404 Commit Not Found. Request URI: https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab/repository/commits/18-1-stable-ee"}
2025-05-27 13:17:07.512279 I [dry-run] ReleaseTools::Slack::ReleaseJobEndNotifier -- Posting slack message -- {job_type: "Create release environment", status: :failed}
  • Output when early_branch_creation is disabled
 
bundle exec rake 'monthly:initial_rc:release_environment'
2025-05-27 13:16:25.764604 W ReleaseTools::Feature -- Unleash::Client is disabled! Will only return default (or bootstrapped if available) results!
2025-05-27 13:16:25.764997 W ReleaseTools::Feature -- Unleash::Client is disabled! Metrics and MetricsReporter are also disabled!

  1. Pipeline creation/configuration testing

    • I disabled other jobs in the initial_rc stage so that they will only echo messages to focus on the release environment creation (commit)

    With MONTHLY_RELEASE_PIPELINE=true

    With MONTHLY_RELEASE_PIPELINE='initial_rc'

    image.png

    Pipeline

    image.png

    Pipeline

  2. Pipeline execution testing

    Step Link/Screenshots

    Manually create a pipeline in the test branch with variables:

    • TEST = "true"
    • SLACK_TEST = "true"
    • MONTHLY_RELEASE_PIPELINE = "initial_rc"

    image.png

    Pipeline

    Trigger the monthly_release_initial_rc:start job

    image.png

    monthly_release_initial_rc:release_environment ran after tag job completed

    image.png

    Creation of the release environment is triggered. It failed, though, since the 18-1 stable branch does not exist yet.

    image.png

    link

    ReleaseJobEndNotifier class is also run as part of the job (dry-run mode)

    image.png

    link

Content

  • Add a new job to the monthly_release:initial_rc stage to create the release environment
  • Add a new class to trigger the release environment creation and send Slack notification at the end
  • Move the feature flag check for skip_release_environment_creation from the release:tag rake task
  • Add a new job to trigger release environment creation separate from tagging

Author Check-list

  • Has documentation been updated?
Edited by Mawreen Dela Cruz

Merge request reports