Updates to the release process on how we carry out the QA task for a given release

Summary

• Release QA Task A Release QA task gets created once per release and gets rolled forward. We use the same ticket to track all the Feature and bugs that we have to verify for every release candidate.
• This is to make sure we do not miss anything for a release and we have a single place to track QA items.
• When we deploy a new RC we update the Release QA task to contain that RC e.g. 10.8 => RC13This way we know which RC we are on for that given release.
• The Feature assurance list comes from the kick off doc. This is the make sure that we link all the product requirements to what is begin released.
  • https://docs.google.com/document/d/1ElPkZ90A8ey_iOkTvUs_ByMlwKK6NAB2VOK5835wYK0/edit
  • Release Managers will use the content in the kickoff document to populate the Feature Assurance section.
• Tracking all QA items in the release As we check off and validate QA items, when a new Release Candidate is created the QA items from that Release Candidate are added as additional items to the Release QA Task. The final RC QA task to contain everything we have in the release.
  • https://docs.google.com/presentation/d/1a1fiw-464vuehiRlGCGbo2S2FtN9Ct5p3SfplvEFTSc/edit#slide=id.g30f1df8898_0_15
  • A Release Manager with the help of a Quality Engineer will populate the additions for each Release Candidate in the QA Release Task. They will also be responsible in mentioning individuals explicitly in comments to validate and check off the QA items.
  • This also helps with making sure we don't have un-validated bug fixes (unchecked validations in QA release tasks) Once ticket to track everything.
• Simplified workflow Instead of tracking multiple QA Issues for a single release where things can fall off the cracks we keep things lean / lightweight and effective. Challenges we had before:
  • Items raised in a previous RC QA Issue were not properly carried over / copied into the the current RC QA Issue.
  • Conversational context maybe lost when a regression theme is correlated in multiple RC builds which spanned multiple issues.

Release task format changes

• Change to use one QA task for a monthly release
  • Change the name to Monthly Release from RC1 QA task
  • Added sections for each new RC release candidate to capture bug fixes and raise regressions
  • Updated instructions to reflect what we are doing in 10.8 and 11.0
  • Add instructions to rename RC# in the QA task
  • Add section for Automated QA which will be taken on by the Quality team
  • Remove the use of snippets
• Patch and Security release QA task
  • Change the name to Patch and Security from QA task
  • Patch and security release have back-port validations and normally only have one iterations with no rolling RCs

Clarification on Quality Engineer role in a monthly release

• Assists RMs in updating the QA issue
• Update the RC information on new changes that needs testing
• Run automated tests
• Post automated test results
• Assists in pinging the team on the QA task to validate results on staging and production where appropriate