Commit 733ec0f8 authored by John Skarbek's avatar John Skarbek Committed by Reuben Pereira
Browse files

Documents QA for backports

parent d92164cc
......@@ -79,6 +79,7 @@ general process overview and specific documentation for different stakeholders.
- [Self-Managed - How to fix a broken stable branch](general/how-to-fix-broken-stable-branch.md)
- [How to resolve failing QA tests](runbooks/resolving-qa-failures.md)
- [How to drain Canary](general/deploy/canary.md#how-to-stop-all-production-traffic-to-canary)
- [Running QA for Backports](runbooks/backport-qa-testing.md)
## Glossary
- [Deployment related terminology](general/deploy/glossary.md)
......
......@@ -4,3 +4,4 @@
- [Variables!](runbooks/variables.md)
- [How to rollback a deployment](runbooks/rollback-a-deployment.md)
- [Fix branch divergence](runbooks/fix-branch-divergence.md)
- [Running QA for Backports](runbooks/backport-qa-testing.md)
# QA Testing Backports
## Reasoning
Currently we do not have a set of servers/environments that can easily install
older versions of GitLab that extend beyond our current patching policy. This
means backports have no place to be installed. Therefore, we rely on QA
pipelines to determine if the proposed package is safe to be released. Note
that we are not testing the results of a built package with this procedure, but
instead testing builds that occur from our pipelines.
## Procedure
### Build the QA Image
1. **Prior to merging the preparation branch**
1. Wait till the `package-and-qa` job becomes available.
* It is a manual job, so it won't start automatically. We MUST specify
variables before starting it.
* See [Extra Information](#extra-information) if `package-and-qa` starts
or is unavailable
1. Click the job name to get to the screen to set variables, set the following
variable:
* `OMNIBUS_BRANCH`: to the name of the matching stable branch of
`omnibus-gitlab` for the proposed release. Example, if working on a
backport for 14.1, you would use `14-1-stable`.
* :warning: Do no click the Play button, as this will not present the screen
for providing variables :warning:
1. Start the job. This job takes a few hours.
1. Wait till, minimally the job `Trigger:gitlab-docker` has completed prior to moving on.
### Execute QA
1. On the [gitlab-qa](https://gitlab.com/gitlab-org/gitlab-qa) project, create a
test branch from the current latest default branch.
1. Navigate to [`https://gitlab.com/gitlab-org/gitlab-qa/-/pipelines`](https://gitlab.com/gitlab-org/gitlab-qa/-/pipelines)
1. Start a new Pipeline using your newly created test branch and the following variables:
* `QA_IMAGE`: This is the name of the image and tag associated with the
GitLab build, example:
`registry.gitlab.com/gitlab-org/gitlab/gitlab-ee-qa:14-1-stable-ee-patch-6`
* `RELEASE`: This is the name of the docker container pushed by the
`Trigger:gitlab-docker` job from the `package-and-qa` downstream pipeline, it SHOULD be:
`registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:<sha>`.
The `sha` SHOULD match that of the latest commit in our preparation
branch.
* See [Extra Information](#extra-information) for a bit more fun notes of
additional variables we may need.
1. This triggers a pipeline in the
[`omnibus-gitlab-mirror`](https://gitlab.com/gitlab-org/build/omnibus-gitlab-mirror)
project.
1. Notify Quality of this pipeline such that they can monitor for any issues.
1. This pipeline will take a few hours.
1. If all is well, QA should sign off on the release issue associated with the
backport indicating that they approve that Release Managers can proceed to
release the build.
## Extra Information
* If the preparation MR is merged prior to running this pipeline, we can still
perform the above process. If new changes need to be picked, they should be
done in a new branch and QA run prior to merge to avoid repeating this lengthy
process multiple times.
* If a new branch needs to be created for any reason, avoid the use of the
character `/` in the branch name as not all scripts properly manage this and
will fail to create a Docker image with a usable tag.
* If the `package-and-qa` job is started without the variable set, we'll need to
create a new commit or branch. There does not exist a way to set the variable
after the job has begun.
* The variable `RELEASE` can be confirmed by navigating to the triggered
pipeline and looking for the job `Trigger:gitlab-docker`. Browse the logs and
then view the raw logs (the data we want is in `stderr`). Scroll to the
bottom and we'll see a message from docker that indicates it pushed an image.
The tag should match the `sha` of the commit of our branch in the
`gitlab-org/gitlab` project.
* Sometimes the `package-and-qa` job will start automatically, this is when the
latest commit contains QA only changes. This is unfortunate, but this
requires a temporary commit to be created that configures the CI rules for the
`package-and-qa` job to be temporarily modified to add a `when: manual` rule.
Do not commit this into the stable branch! This [configuration can be found
here](https://gitlab.com/gitlab-org/gitlab/-/blob/b829a07d08a280bd5f495d6a727d5f2ed99e3bc3/.gitlab/ci/rules.gitlab-ci.yml#L746-751)
* We lack permissions on the `gitlab-qa` project to be able to create a pipeline
on the current default branch. Hence we create a test branch.
* As time progressed, our default branch changed from `master` to `main` and in QA
we need to specify which default branch should be used for testing. If we are
testing versions 14.1 and 14.0, we must set variable `QA_BRANCH` to `main`
when executing a pipeline in the `gitlab-qa` project.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment