FY22 Q1 KR: Reduce setup time for Merge Request regression tests by 50% from 40 to 20mins => 65%
The following needs to be done to achieve this KR:
-
Analyse whether review-apps can be used for end-to-end tests. YES - gitlab-org/gitlab!55583 (merged) -
Fix tests failing only on Review Apps => 100% complete- gitlab-org/gitlab#30448 (closed) -
Figure out why review-qa-smoketests still sometimes fail=> 50%(gitlab-org/gitlab#323741 (closed)) -
Analyse the gap between using review-qa-allandpackage-and-qa- How can this be addressed optimally? -
Speed up package-and-qa- gitlab-org/omnibus-gitlab#5429 (closed) -
Make it possible to skip tests that can't be run on Review Apps - https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/821 => 100% -
Test running Review Apps as a hybrid architecture with Gitaly off k8s in progress, 10% -
Make review-qa-allmandatory forqafolder specific MRsin progress, 80% -
Set up monitoring/alerting so that we don't have unexpected failures due to resource constraints in Review Apps - https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/834 in progress, 80% -
Document the new pipeline/job related details and communicate the same. -
Measure the pipeline/job success regularly.
** Stretch Goals:
- Can the
security fixedrun this as well?
Monitoring
Edited by Ramya Authappan