12.0.2 QA Issue
Important
Please use pre.gitlab.com for QA testing. You can login with your google account (if you are part of GitLab Inc.). Do note that pre.gitlab.com has no seed data.
Process
Each engineer validates and checks off each of their assigned QA task(s).
- Check off each Merge Request changes that you've tested successfully and note any issues you've created and check them off as they are resolved.
- If a problem is found:
- Create an issue for it and add a sub bullet item under the corresponding validation checklist task. Link the issue there.
- Add the severity label
- Raise the problem in the discussion and tag relevant Engineering and Product managers.
- If a regression is found:
- Create an issue for it
- Add the severity label and the regression label
- Raise the regression in the discussion and tag relevant Engineering and Product managers.
General Quality info can be found in the Quality Handbook.
Note: If you are assigned tasks outside your normal work hours, you're not expected to work overtime. Please complete the tasks as soon as possible during your normal work hours.
Deadline
QA testing on staging.gitlab.com for this issue should be completed by 2019-06-26 10:07 UTC. After this deadline has passed, Release Managers will proceed with the canary and production deployment.
If the deadline has passed, please perform your task as soon as possible anyway (during your normal work hours). It's important that the testing is performed, even if deployment has proceeded to a later stage.
Merge Requests tested in 12.0.2
Community contribution
Community contribution- @stanhu | Fix missing API notification flags for Microsoft Teams 1st contribution Community contribution api ~"services"
Plan ~"Plan"
- @mdelaossa | Do not rewrite relative links for system notes ~"P2" ~"Plan" ~"S2" backend ~"bug" devopsplan ~"group::team planning" regression regression:12.0
- @smcgivern | Fix label serialisation in issue and note hooks ~"Plan" backend ~"bug" regression regression:12.0 ~"webhooks"
- @acroitor | Add documentation on epic add/remove child relations quick actions ~"Documentation" ~"Plan" devopsplan epics ~"group::team planning" quick actions
- @smcgivern | Fix notes email with group-level notification email ~"P2" ~"Plan" ~"S2" backend ~"bug" regression regression:12.0
- @weimeng | Take into account events created before milestone start ~"Plan" auto updated ~"bug" regression ~"support-fix"
Create ~"Create"
- @pslaughter | Fix IDE commit to use start_ref ~"Create" Deliverable ~"P1" ~"Pick into 11.11" ~"S2" ~"bug" depth frontend ~"group::editor" internal customer missed-deliverable missed:11.11 ~"web ide"
- @mdelaossa | Fix ElasticSearch search results pagination ~"Create" backend ~"bug" ~"elasticsearch" ~"group::editor" regression:12.0
Manage ~"Manage"
- @gitlab-crowdin-bot | New Crowdin translations ~"Manage" ~"Pick into 11.11" frontend ~"internationalization"
- @lbennett | Master i18n ~"Manage" ~"Pick into 11.11" frontend ~"internationalization"
Verify ~"Verify"
- @filipa | Adds missing class in collapsible sections ~"Verify" backend ~"bug" frontend
- @sarahghp | Review Toolbar: Add closing tag ~"Verify" ~"bug" frontend
Geo ~"Geo"
- @ashmckenzie | Gitlab::JsonCache#parse_value ensure string ~"Geo" ~"P1" ~"backstage" ~"bug" ~"workflow::In dev"
- @dbalexandre | Include the GitLab version in the cache key for Gitlab::JsonCache ~"Geo" ~"bug" ~"devops::enablement" groupgeo
Gitaly ~"Gitaly"
- @GitalyBot | Upgrade Gitaly to v1.47.0 ~"Gitaly" ~"backstage"
- @zj-gitlab | Feature flag default on catfile cache ~"Gitaly" ~"backstage"
Monitor ~"Monitor"
- @eread | Edit new Zoom call link content ~"Documentation" ~"Monitor" devopsmonitor docs-only ~"feature" ~"group::health"
Secure ~"Secure"
- @markrian | Work around missing pagination headers ~"Secure" ~"backstage" devopssecure
- @theoretick | Move approval MR rule report_type migration CE ~"Secure" ~"backstage" single codebase
- @theoretick | Backport approval MR rules report_type migration to CE ~"Secure" ~"backstage" single codebase
- @vzagorodny | Fix typos, grammar and wording for SAST and DS report JSON docs ~"Documentation" ~"Secure" ~"Secure::Static and Dynamic Analysis" backend ~"backstage" devopssecure ~"docs-fix" ~"docs:fix" ~"feature"
- @brytannia | Revert "Merge branch 'dependency-list-feature-flag-disabled-by-default' into 'master'" ~"Secure" ~"Secure::Software Composition Analysis" ~"backstage" devopssecure
- @axil | Remove Gemnasium dead link from docs ~"Documentation" ~"Secure"
database
database- @yorickpeterse | EE: Backport the EE schema to CE ~"backstage" database single codebase
uncategorized ~"uncategorized"
- @dosuken123 | Revert concurrent pipeline creation for pipeline schedules ~"P2" ~"S2" ~"bug"
- @stanhu | Omit issues links in merge request entity API response devopscreate merge requests ~"performance"
- @stanhu | Silence backup warnings when CRON=1 in use ~"bug" devopscreate regression:12.0
Automated QA for 12.0.2
No QA job could be found for this release!
You will need to set up a dedicated environment for 12.0.2 by following the following steps:
Prepare the environments for testing the security fixes
Instructions to prepare environment
- In Google Cloud Console (access to this
should have been granted during on-boarding), create a new VM instance (in the
gitlab-internal
project) from theqa-security-1cpu-3-75gb-ram-ubuntu-16-04-lts
instance template for each version of GitLab. - Find the
.deb
package to install:- First find the pipeline for the
12.0.2+ee.0
tag in the pipelines page. - Then on the pipeline page, click the
Ubuntu-16.04-staging
job in theUpload:gitlab_com
stage (or theStaging_upload
stage for versions prior to 11.5), you will need the job ID later.
- First find the pipeline for the
- Install the
.deb
package from the job artifact:- SSH into the VM via the GCP console.
- Create a
install-gitlab.sh
script in your home folder:TEMP_DEB="$(mktemp)" GITLAB_PACKAGE="https://dev.gitlab.org/api/v4/projects/gitlab%2Fomnibus-gitlab/jobs/${JOB_ID}/artifacts/pkg/ubuntu-xenial/gitlab-ee_${GITLAB_VERSION}-ee.0_amd64.deb" curl -H "PRIVATE-TOKEN: $DEV_TOKEN" "$GITLAB_PACKAGE" -o "$TEMP_DEB" && sudo dpkg -i "$TEMP_DEB" rm -f "$TEMP_DEB"
-
$DEV_TOKEN
needs to be set with adev.gitlab.org
personal access token so that the script can download the package -
$JOB_ID
needs to be set with theUbuntu-16.04-staging
job ID -
$GITLAB_VERSION
needs to be set with the version (without the-ee
prefix, e.g.11.4.10
).
-
- Change the script's permission with
chmod +x install-gitlab.sh
. - Run the script with
./install-gitlab.sh
. - Once GitLab installed, set the
external_url
in/etc/gitlab/gitlab.rb
withsudo vim /etc/gitlab/gitlab.rb
. You can find the VM's IP in the GCP console. - Reconfigure and restart GitLab with
sudo gitlab-ctl reconfigure && sudo gitlab-ctl restart
. - You may need to wait a few minutes after the above command finishes before the instance is actually accessible.
- Set the
root
's user password:- Visit http://IP_OF_THE_GCP_VM and change
root
's password. - Once the environments are ready, capture the information to add to the QA issue.
- Visit http://IP_OF_THE_GCP_VM and change
Automated QA
Instructions to perform automated QA
-
(Optional) If the QA Docker image doesn't exist, you will need to build it manually on your machine, e.g.
# In gitlab-ee › git fetch dev › git checkout v12.0.2-ee › cd qa › docker build -t dev.gitlab.org:5005/gitlab/omnibus-gitlab/gitlab-ee-qa:12.0.2-ee .
-
Make sure to export the following environment variables (you can find the token under the
GitLab QA - Access tokens
1Password items)-
$QA_IMAGE
the URL of the QA image -
$QA_ENV_URL
with the URL of the environment where the package has been deployed (usually https://staging.gitlab.com for the current version, andhttp://IP_OF_THE_GCP_VM
for back-ported versions). -
$GITLAB_USERNAME
withroot
. -
$GITLAB_ADMIN_USERNAME
with$GITLAB_USERNAME
. -
$GITLAB_PASSWORD
with the password you've set for theroot
user. -
$GITLAB_ADMIN_PASSWORD
with$GITLAB_PASSWORD
. -
$GITHUB_ACCESS_TOKEN
with a valid GitHub API token that can access the https://github.com/gitlab-qa/test-project project -
$DEV_USERNAME
with yourdev
username -
$DEV_TOKEN
with a validdev
personal access token that has theread_registry
scope
› export QA_IMAGE="dev.gitlab.org:5005/gitlab/omnibus-gitlab/gitlab-ee-qa:12.0.2-ee" › export QA_ENV_URL="<QA_ENV_URL>" › export GITLAB_USERNAME="root" › export GITLAB_ADMIN_USERNAME="$GITLAB_USERNAME" › export GITLAB_PASSWORD="<GITLAB_PASSWORD>" › export GITLAB_ADMIN_PASSWORD="$GITLAB_PASSWORD" › export GITHUB_ACCESS_TOKEN="<GITHUB_ACCESS_TOKEN>" › export DEV_USERNAME="<DEV_USERNAME>" › export DEV_TOKEN="<DEV_TOKEN>"
-
-
Update
gitlab-qa
if needed› gem install gitlab-qa
-
Log into the
dev
container registry› docker login --username "$DEV_USERNAME" --password "$DEV_TOKEN" dev.gitlab.org:5005
-
Automated QA completed. QA can be parallelized manually (for now):
# Tab 1: This should take approximately 4.5 minutes › gitlab-qa Test::Instance::Any $QA_IMAGE $QA_ENV_URL -- qa/specs/features/api/ qa/specs/features/login/ qa/specs/features/merge_request/
# Tab 2: This should take approximately 6 minutes › gitlab-qa Test::Instance::Any $QA_IMAGE $QA_ENV_URL -- qa/specs/features/project/
# Tab 3: This should take approximately 5 minutes › gitlab-qa Test::Instance::Any $QA_IMAGE $QA_ENV_URL -- qa/specs/features/repository/
-
Post results as comments of this issue
-
Create
Automation Triage RELEASE_MAJOR_VERSION RC#
issues for all the automated QA failures (with failures logs + screenshots) and link it to this issue
Coordinate the Manual QA validation of the release
Click for details
- Notify the Security Engineer to verify the security fixes for the release.
- The manner in which the security fixes are verified can be done in two ways.
- By the Quality Engineer executing the validation with close collaboration and guidance from the Security Engineer.
- By the Security Engineer executing the validation with the Quality Engineer monitoring the steps.
- Note: When encountered with deadline and resource constraints, the work should be assigned for efficiency. Security Engineer should own verifying complex security validations while Quality Engineer is encouraged to help out with simpler validations. However it is important that the Security team signs off on the result of the validation.
- The manner in which the security fixes are verified can be done in two ways.
- Ensure that all the items for validation are validated and checked off before moving forward.
- Hand off the release assignment.
- Once all the validation is completed, Quality Engineer un-assigns themselves from the release issue leaving only the Security Engineer and the Release Manager.
/cc @gl-quality