2018-06-01 11.0 QA Task RC6

Deadline

  • The deadline to which the first release candidate (RC1) moves on from staging environment is 24 hours after the completion each deploy to staging.
  • The deadline to which subsequent release candidates moves on from staging environment is 12 hours after the completion each deploy to staging.
  • The deadline to which the QA Task is closed out is 24 hours (1 working day) after the final deploy to production.

Note: For Release Managers, for every new release candidate, update the time here to reflect the latest release candidate deploy.

QA testing on staging.gitlab.com should be completed by 2018-06-08 11:00 UTC. After this deadline has passed, Release Managers will proceed with the production deployment.

Tasks

General Quality info can be found at the Quality Handbook.

You can use the QA Checklist to ensure you've tested critical features.

  1. List features that need to be tested, with a link to the related issue.
  2. Check off any feature you've tested successfully.
  3. If a problem is found, update the item adding the link to the related issue, and the severity of it
  4. Check them off as they are resolved

Bugs & Changes tested

Check off any features you've tested successfully and note any issues you've created and check them off as they are resolved.

If there are regressions please raise them in the discussion and add an entry for it to be validated in the following RC.

RC1 - MR list with ordering by team

Discussion ~"Discussion"

Platform ~"Platform"

CI/CD ~"CI/CD"

Geo ~"Geo"

Gitaly ~"Gitaly"

Monitoring ~"Monitoring"

Security Products ~"Security Products"

frontend frontend

RC2

RC3

RC4

RC5

RC6

Discussion ~"Discussion"

Platform ~"Platform"

CI/CD ~"CI/CD"

Geo ~"Geo"

Gitaly ~"Gitaly"

Monitoring ~"Monitoring"

Security Products ~"Security Products"

frontend frontend

database database

uncategorized ~"uncategorized"

Automated QA

Note: For Quality Engineers, for every release versions run Gitlab QA on staging and post the results.

Please post the results of the gitlab-qa automated QA tests below.

The credentials are in 1Password, look for GitLab QA.

Run

GITLAB_USERNAME=your_username GITLAB_PASSWORD=your_password gitlab-qa Test::Instance::Any EE latest https://staging.gitlab.com

RC1 results

Finished in 13 minutes 12 seconds (files took 0.58786 seconds to load)
19 examples, 7 failures

Failed examples:

rspec ./qa/specs/features/merge_request/create_spec.rb:3 # creates a merge request user creates a new merge request
rspec ./qa/specs/features/api/users_spec.rb:16 # API users when authenticated submit request with a valid user name
rspec ./qa/specs/features/api/users_spec.rb:10 # API users when authenticated get list of users
rspec ./qa/specs/features/api/users_spec.rb:25 # API users when authenticated submit request with an invalid user name
rspec ./qa/specs/features/project/add_secret_variable_spec.rb:3 # secret variables support user adds a secret variable
rspec ./qa/specs/features/merge_request/rebase_spec.rb:3 # merge request rebase rebases source branch of merge request
rspec ./qa/specs/features/project/deploy_key_clone_spec.rb:18 # cloning code using a deploy key user sets up a deploy key to clone code using pipelines

RC5 results

Finished in 15 minutes 39 seconds (files took 0.26677 seconds to load)
19 examples, 6 failures

Failed examples:

rspec ./qa/specs/features/api/users_spec.rb:16 # API users when authenticated submit request with a valid user name
rspec ./qa/specs/features/api/users_spec.rb:25 # API users when authenticated submit request with an invalid user name
rspec ./qa/specs/features/api/users_spec.rb:10 # API users when authenticated get list of users
rspec ./qa/specs/features/project/add_secret_variable_spec.rb:3 # secret variables support user adds a secret variable
rspec ./qa/specs/features/project/deploy_key_clone_spec.rb:18 # cloning code using a deploy key user sets up a deploy key to clone code using pipelines
rspec ./qa/specs/features/merge_request/create_spec.rb:3 # creates a merge request user creates a new merge request

RC# results

Create a new section for each new release candidate, update the section with changes in the release and update the RC# in the issue description

Post the result of the test run here

/cc @gl-product

Edited by Mayra Cabrera