Automate the remaining QA tasks in the security release (patch release process)
Currently its quite an expensive and manual process to run Gitlab-QA for a patch release
See https://gitlab.com/gitlab-org/release/tasks/issues/498#automated-qa-for-1142
Proposal
There are three things to do here:
- Update
gitlab-provisioner
to use the QA docker image fromdev
instead of from.com
so that security releases can be tested (otherwise thegitlab-qa
job fails because the image cannot be found: https://gitlab.com/gitlab-org/distribution/gitlab-provisioner/-/jobs/169671539). - Put the link to the QA job in the QA issues (security and non-security ones).
- Keep the Terraform deployment running to allow manual QA.
For 1., the steps are as follows:
-
Change gitlab-provisioner
to log into thedev
registry prior to running the QA job => gitlab-org/distribution/gitlab-provisioner!22 (merged) -
Change the HA-Validate-Trigger
job to pass thedev
image address instead of the Docker Hub one, e.g.dev.gitlab.org:5005/gitlab/omnibus-gitlab/gitlab-ee-qa:11.8.1-ee
instead ofgitlab/gitlab-ee-qa:11.8.1-ee
=> gitlab-org/omnibus-gitlab!3107 (merged) - Enjoy
🍿
For 2., the steps are as follows:
-
Find the omnibus-gitlab
pipeline for the given version, e.g.GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines?ref=11.9.0%2Brc6.ee.0
- Getting the
omnibus-gitlab
tag from the release version can be done withversion.to_omnibus(ee: true)
.
- Getting the
-
Find the HA-Validate-Tagged
job in this pipeline by looping over the pipelines' jobs and select the first job with nameHA-Validate-Tagged,
, e.g.GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/pipelines/106960/jobs
-
Fetch the job's trace, e.g. GET https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/jobs/4122266/trace
-
Search for Waiting for downstream pipeline status: https://gitlab.com/gitlab-org/distribution/gitlab-provisioner/pipelines/<ID>
in the trace -
We could either display this pipeline URL, or -
Continue the search for the gitlab-qa
job (this time it will be on GitLab.com) -
Once the job found, put the link in the QA issue template before the issue is created -
Enjoy 🍿
=> gitlab-org/release-tools!577 (merged)
For 3., the steps are as follows:
- Don't automatically stop the GitLab deployment to allow manual QA testing.
- Maybe cleanup deployments automatically after 2 or 3 days (using a schedule)?
Original proposal
- Make sure the change summary automation works for the patch release
- Automate the provisioning of environments using new review apps
- Automate the test automation pipelines for GitLab QA against all the ported version environments
Edited by Rémy Coutable