Internal release tools testing for the maintenance policy extension
The work towards extending the maintenance policy is in progress, however, modifications for the release toolings are already in place. The purpose of this issue is to perform internal testing to ensure release managers have the required tooling to perform patch releases for three versions.
This internal testing is a 'dry-run' in terms that no patch release version should be tagged or published.
Exit criteria
-
Guidance is provided when merging merge requests into stable branches -
type::bug
label is enforced -
package-and-test
pipeline execution is encouraged -
Merge requests outside the policy throw a warning
-
-
Tooling -
Patch release pressure reflects the state of the MRs merged into stable branches -
Patch release task issue is generated -
Blog posts accounts for bug fixes merged into stable branch -
The steps in the issue template account for three versions
-
Requirements
Merge requests targeting 15-9-stable-ee
should be prepared.
-
For the merge requests, identify typos throughout the app:
List
- Doc changes:
-
doc/development/rake_tasks.md:261:66
:seperated
should beseparated
-
doc/development/pages/index.md:227:8
:specfic
should bespecific
-
- Spec changes:
-
spec/models/error_tracking/project_error_tracking_setting_spec.rb:418:61
:appers
should beappears
-
spec/models/error_tracking/project_error_tracking_setting_spec.rb:305:46
:relase
should berelease
-
spec/models/container_registry/event_spec.rb:137:20
:respository
should berepository
-
- App changes:
-
app/models/container_registry/event.rb:59:15
:existance
should beexistence
-
app/models/ci/job_artifact.rb:373:13
:overriden
should beoverridden
-
app/models/ci/job_artifact.rb:377:13
:overriden
should beoverridden
-
-
Prepare merge requests targeting 15-9-stable-ee
. Merge requests should contain simple changes, for example: documentation updates, fix for flaky failures, or small backend and frontend changes.-
Doc changes - gitlab-org/gitlab!112612 (merged) -
Spec changes - gitlab-org/gitlab!112613 (merged) -
App changes - gitlab-org/gitlab!112615 (closed)
-
-
Create backport merge request to the respective stable branch.
Merge request process
✅ With a properly set merge request
- Tested on 2023-02-23
- MR: gitlab-org/gitlab!112615 (closed)
Tests:
-
Ensure the merge request has the type::bug
label -
Verify Danger should not fail https://gitlab.com/gitlab-org/gitlab/-/jobs/3824560803 -
Ensure the package-and-test
pipeline is automatically executed -
Ensure the Reviewer Roulette is displayed as part of Danger -
Ensure the package-and-test
execution is encouraged
Package-and-qa message (gitlab-org/gitlab!112615 (comment 1287297821)) |
---|
Observations
Small details were found on #2839 (comment 1287312560). Fixes were created:
- Skip danger check on documentation MRs (gitlab-org/gitlab!112751 - merged)
- Remove pick into x.y message from Danger (gitlab-org/gitlab!112740 - merged)
- Split QA into a dedicated section (gitlab-org/gitlab!112742 - merged)
- Introduce a stable branch MR template (gitlab-org/gitlab!112736 - merged)
✅ With a merge request using the wrong labels
- Tested on 2023-02-23
- MR gitlab-org/gitlab!112613 (merged)
-
Add the type::feature
label and verify that Danger fails -
Add the pipeline:expedite
label and verify that Danger fails
Danger job | Labels | Danger error messages |
---|---|---|
https://gitlab.com/gitlab-org/gitlab/-/jobs/3824942095 | ||
https://gitlab.com/gitlab-org/gitlab/-/jobs/3824992959 |
Package-and-test
pipeline
✅ General
Tests
-
Danger pings the author and states the package-and-test
pipeline should be executed, and failures should be reviewed by the SET counterpart
Package-and-test |
Danger job | Warning message |
---|---|---|
https://gitlab.com/gitlab-org/gitlab/-/pipelines/784674631 | gitlab-org/gitlab!112615 (comment 1287297821) |
Observations:
- The Danger message could be highlighted as an action that needs to be taken. gitlab-org/gitlab!112855 (merged)
- The message states "See the section below for more details. But this could be ambiguous since are many sections, the message should state what Danger section should be read. gitlab-org/gitlab!112855 (merged)
✅ When the package-and-test
is not present
-
Guidance about the requirement of package-and-test
should be indicated by Danger. -
Danger should fail
Observations:
- The error message states the
package-and-test
needs to be manually triggered, this is not the case on stable branches because the pipeline is automatically executed. gitlab-org/gitlab!112855 (merged)
MR | Danger job | Danger error message |
---|---|---|
gitlab-org/gitlab!112743 (closed) | https://gitlab.com/gitlab-org/gitlab/-/jobs/3825789810 |
✅ Status: canceled
-
Danger pings the author and states the package-and-test
pipeline should be executed, and the SET counterpart should review failures -
Danger should fail
package-and-test |
Danger job | Error | Message |
---|---|---|---|
https://gitlab.com/gitlab-org/gitlab/-/pipelines/795615508 | https://gitlab.com/gitlab-org/gitlab/-/jobs/3872422308 |
Observations
- Danger didn't fail because it is assuming the
bridge
status rather than the downstream pipeline status gitlab-org/gitlab!113272 (diffs)
✅ Status: success
-
Danger pings the author and states the package-and-test
pipeline should be executed, and the SET counterpart should review failures
package-and-test |
Danger job | Warning | Message |
---|---|---|---|
https://gitlab.com/gitlab-org/gitlab/-/pipelines/793261571 | https://gitlab.com/gitlab-org/gitlab/-/jobs/3858303015 | N/A |
Observations:
- The message states 'manually triggered'. On stable branches, this message is automatically triggered so the message is inaccurate. It may also be a good idea to refer engineers to the
Package-and-test
section gitlab-org/gitlab!112855 (merged)
✅ Status failed
-
Danger pings the author and states the package-and-test
pipeline should be executed, and the SET counterpart should review failures
package-and-test |
Danger job | Warning | Message |
---|---|---|---|
https://gitlab.com/gitlab-org/gitlab/-/pipelines/784674631 | https://gitlab.com/gitlab-org/gitlab/-/jobs/3858303015 |
✅ Documentation and flaky merge requests
-
Ensure the package-and-test
pipeline is not encouraged on documentation MRs -
Ensure the package-and-test
pipeline is not encouraged on merge requests withflaky::failure-test
label
Documentation MRs | MRs with flaky::failure-test
|
|
---|---|---|
Danger message | ||
Danger job | https://gitlab.com/gitlab-org/gitlab/-/jobs/3873200250 | https://gitlab.com/gitlab-org/gitlab/-/jobs/3873226158 |
Merge request | gitlab-org/gitlab!112612 (merged) | gitlab-org/gitlab!112613 (merged) |
Observations:
- Is it possible to skip the
package-and-test
execution for documentation changes?- Technically should be possible but this is a nice-to-have. Added to the nice to have section.
- The
package-and-test
pipeline was actually encouraged on both, merge requests with documentation changes and merge requests fixing flaky failures. Fixed by gitlab-org/gitlab!113431 (merged)
✅ Merge requests targeting versions outside the policy
-
Ensure a Danger message is listed specifying the version is outside the policy.
MR (gitlab-org/gitlab!113436 (comment 1299441328)) |
---|
Backport MRs:
- Skip danger check on documentation MRs (gitlab-org/gitlab!112751 - merged)
- Remove pick into x.y message from Danger (gitlab-org/gitlab!112740 - merged)
- Split QA into a dedicated section (gitlab-org/gitlab!112742 - merged)
- Introduce a stable branch MR template (gitlab-org/gitlab!112736 - merged)
- Highlight package-and-qa Danger message (gitlab-org/gitlab!112855 - merged)
- Fail danger if package-and-test pipeline is can... (gitlab-org/gitlab!113272 - merged)
- Skip package-and-test message on trivial MRs (gitlab-org/gitlab!113431 - merged)
- Prevent QA check on stable branches (gitlab-org/gitlab!113281 - merged)
- Skip Danger failure when package-and-test is be... (gitlab-org/gitlab!113564 - merged)
Backported on gitlab-org/gitlab!113549 (merged) and gitlab-org/gitlab!113871 (merged)
Test with the backports merged and backported into the stable branch #2839 (comment 1299525695)
✅ Path release pressure
-
Merge the merge requests into the stable branch -
Ensure the patch release pressure is increased
Status | MR gitlab-org/gitlab!112612 (merged) | MR gitlab-org/gitlab!112613 (merged) | Patch release pressure (general) | Patch release pressure (by version) |
---|---|---|---|---|
severity1 | severity2 | |||
severity1 | severity3 | |||
severity3 | severity4 | |||
N/A | N/A |
✅ Tooling verification
-
Execute /chatops run release prepare
https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/9423218 -
Ensure only the release task issue and the blog post are created -
Release task issue: gitlab-org/release/tasks#5183 (closed) -
Blog post: gitlab-com/www-gitlab-com!120963 (closed)
-
-
Corroborate the blog post created contains the merge requests -
Follow the steps listed and ensure they make sense
Observations
- After creating the blog post, there is no easy way for this one to be updated if merge requests are merged after that. The workaround is to close the blog post and re-trigger the
/chatops run release prepare
command
✅ With a failure on a stable branch
-
Ensure a message is posted on #releases
-
Ensure an issue is opened in https://gitlab.com/gitlab-org/release/tasks
Nice to haves
- Skip the
package-and-test
pipeline on documentation MRs - gitlab-org/quality/quality-engineering/team-tasks#1691 - Skip the message suggesting a
type::
label should be added. This is taken care of by Danger. #2894 - Checking stable branches for projects under Managed Versioning is super manual, it would be nice to have some automation for this (this will be a nice to have though) #2895
- Define a patch release threshold #2896 (closed)
- Automatically alert RM about S1/S2 unreleased merge requests. #2897 (closed)
- After creating the blog post, there is no easy way for this one to be updated if merge requests are merged after that. The workaround is to close the blog post and re-trigger the
/chatops run release prepare
command #2898 (closed)
Edited by Mayra Cabrera