FY22-Q2 KR - Make review-qa-smoke mandatory for every MR => 94%
Continue efforts on #10557 (closed)
Based on failure analysis from gitlab-org/gitlab#323999 (closed), we identified the following action items that can improve review-qa-smoke
reliability and enable us to make it mandatory and blocking in all MRs:
-
Test running Review Apps as a hybrid architecture with Gitaly off k8sRun Gitaly on non-preemptible nodes and confirm Gitaly-related failures are resolved100%
-
Retry review-qa-* for all failures once gitlab-org/gitlab!61111 (merged) -
Retry review-deploy
job twice gitlab-org/gitlab!62272 (merged) -
De-duplicate jobs for the same commit id gitlab-org/quality/toolbox!75 (merged) -
Filter out jobs from the report that pass on retry: gitlab-org/quality/toolbox!77 (merged) -
Disallow review-deploy
job to fail gitlab-org/gitlab!61655 (merged)pending deploy being stable, 50%
-
Remove flaky pages_pipeline_spec
andcreate_and_process_pipeline_spec
from:smoke
suite till proved to be stable gitlab-org/gitlab!62639 (merged) -
RewriteAs the example is still flaky, we decided to exclude it from running increate_issue_spec
to use a different upload method or different upload file that will work in Review App gitlab-org/gitlab!64467 (merged)review-qa-smoke
job in gitlab-org/gitlab!65705 (merged)
Daily reports on review-qa-smoke
failures are generated via https://gitlab.com/gitlab-org/quality/toolbox/-/pipeline_schedules/106780/edit scheduled job and sent to #g_qe_dev
Slack channel.
Edited by Mark Lapierre