Run package-and-qa on master more frequently
Traceability of the validation for gitlab-qa runs against gitlab-org/gitlab@master -> staging -> production is unclear. In an effort to improve this traceability to reduce issue occurrences like https://gitlab.com/gitlab-com/gl-infra/infrastructure/issues/7907
Requirements
- Trigger gitlab-qa image to be built more frequently
- Create a new scheduled pipeline every 2 hours to trigger tests against master - Engineering Productivity
- Validate that executed test sets contain the necessary tests for each stage/group - Counterpart TAEs
- Ensure handbook documents what test sets are run in which environment on given scheduled frequencies.
Future iterative improvements
- Collaborate with Delivery team to ensure deployments to staging contain only green master builds
Proposal
-
Modify gitlab-qa/nightly frequency to be nightly -
Utilize an existing scheduled pipeline to trigger the package-and-qamore frequently against master - gitlab-org/gitlab!17237 (merged) -
Send a Slack notification when a gitlab-qapipeline that run formasterschedules succeed or fail. - tracked in gitlab-org/gitlab#32776 (closed), gitlab-org/gitlab!17796 (merged) -
Adjust test run in package-and-qabased on necessary test sets for execution against master - Test Automation Engineers
Discussed during Quality Engineering Staff meeting on 2019-09-18 - https://docs.google.com/document/d/1mvKwDBCoYqIs9zB_PTUUhUoBKs_bOxtz3x15mNLkUu0/edit?ts=5d824705#bookmark=id.1kt130ch6hzn
Edited by Albert Salim