FY22-Q2 KR - Make review-qa-smoke mandatory for every MR => 94%
Continue efforts on #10557 (closed)
Based on failure analysis from gitlab-org/gitlab#323999 (closed), we identified the following action items that can improve review-qa-smoke reliability and enable us to make it mandatory and blocking in all MRs:
-
Test running Review Apps as a hybrid architecture with Gitaly off k8sRun Gitaly on non-preemptible nodes and confirm Gitaly-related failures are resolved100% -
Retry review-qa-* for all failures once gitlab-org/gitlab!61111 (merged) -
Retry review-deployjob twice gitlab-org/gitlab!62272 (merged) -
De-duplicate jobs for the same commit id gitlab-org/quality/toolbox!75 (merged) -
Filter out jobs from the report that pass on retry: gitlab-org/quality/toolbox!77 (merged) -
Disallow review-deployjob to fail gitlab-org/gitlab!61655 (merged)pending deploy being stable, 50% -
Remove flaky pages_pipeline_specandcreate_and_process_pipeline_specfrom:smokesuite till proved to be stable gitlab-org/gitlab!62639 (merged) -
RewriteAs the example is still flaky, we decided to exclude it from running increate_issue_specto use a different upload method or different upload file that will work in Review App gitlab-org/gitlab!64467 (merged)review-qa-smokejob in gitlab-org/gitlab!65705 (merged)
Daily reports on review-qa-smoke failures are generated via https://gitlab.com/gitlab-org/quality/toolbox/-/pipeline_schedules/106780/edit scheduled job and sent to #g_qe_dev Slack channel.
Edited by Mark Lapierre