Snipped from https://hackmd.io/vdYP7DO5RDatOwG686ez3Q

--

Title : Reducing marge-bot pipeline wall-time on tezos/tezos

Description : This is the end-goal: make the marge-bot pipelines shorter so that devs wait shorter.

Estimated effort (in nb weeks): : --

Associated KRs : CI runs consistently under 20 minutes and cost is not higher that the current one

Dependencies : Axis 1

Part of : https://gitlab.com/groups/tezos/-/milestones/5

Metrics & objective

• Goal metrics (measured by):
  • Projected Average/Worst-case Wall-time of Marge-bot pipelines (M1.1)
  • Recorded Average/Worst-case Wall-time of Marge-bot pipelines (M2.1)
• Guardrail metrics (measured by):
  • Quality assurance level (??)
    • This is very hard to quantify in a measurable manner
  • Projected / recorded AWS cost (M1.2 / M4)

Objectives:

• Goal metrics:
  • Projected average/worst-case wall-time of marge-bot is under 20 minutes/30 minutes
• Guardrail metrics
  • Quality assurance level: should be maintained
  • Recorded / projected AWS cost: should not increase

Task breakdown

• build_x86_64: batch dune calls @(Arvid, Pietro) {1d}
• Increase the number of Tezt jobs @(Arvid, Pietro) {1d}
• script:snapshot_alpha_and_link: combine into build_x86_64 @(Arvid) {2-3d}
• unified_coverage: move to master pipeline on marge-bot pipelines @(Arvid) {2d}
• Flaky tests: make and implement policy @(Arvid, Pietro) {2d}
• Flaky tests: ward off @(Pietro, Arvid) {2d}
• Finish the flakiness detection pipeline @(Arvid) {1d}
• Flaky tests: fix @(Pietro, Arvid) {??}
• Unit tezts: only test modified + revdeps @(Arvid, Pietro) {10d}
• Per job-specific machines as per profile @(Charles, Corentin) {??}
• build_x86_64: investigate dedicated build machine(s) @(Charles, Corentin) {1d}
• build_x86_64: implement dedicated build machine(s) @(Charles, Corentin) {??}
• Figure out what to do with old protocols @(Pietro, arvid) {??}

Deliverables

• A pipeline configuration with shorter average/worst-case marge-bot pipeline wall-times