CI: Rebalance integration tests after Kathmandu freeze
Context
Depends on: !7351 (merged)
Tezt and pytest jobs have become quite unbalanced. This is how it looks after the freeze of Kathmandu in !7351 (merged), on which this MR depends:
This MR:
- Changes
tezt/records/update.ml
so that it fetches jobs from pipelines in batches of 100 instead of 20 (the default). This speeds up the script. - Fixes fetching of artifacts in
tezt/records/update.ml
. It now fetches artifacts using the API. The old implementation broke due to a change in gitlab (the previous URL now shows an html page that the user has to click through to get the artifact). - updates the tezt records and pytest
test-results.xml
from pipeline 746431821 in !7351 (merged) -
pytest
andpytest_old_protocols
are split in too many jobs (16 respectively 20). I reduce it to 10 respectively 8.- the sequential running time of the
pytest_old_protocols
is 102 minutes split over 16 jobs. There is roughly one minute of overhead per job, so 80 minutes of actual work. Assuming perfect balancing, we can run 10 jobs instead and still retain a wall-time of (80/10 + 1) ~9 minutes per job. - the sequential running time of the
pytest
is ~80 minutes split over 20 jobs. There is roughly one minute of overhead per job, so 60 minutes of actual work. Assuming perfect balancing, we can run 8 jobs instead and still retain a wall-time of (60/8 + 1) ~9 minutes per job.
- the sequential running time of the
To note: we're removing all pytests and I expect that we will be finished quite soon (at least, there are open MRs for roughly half of the LOCs left and the remaings LOCs are simple but repetitive tests). In normal times, it would be future-smart to leave some slack, but in this case I think we can allows ourselves to tighten the belt.
Manually testing the MR
Results afterwards.
So, actually it's as unbalanced as before :(
looking good
Forgot to run pytest for old protocols :(
Checklist
-
Document the interface of any function added or modified (see the coding guidelines) -
Document any change to the user interface, including configuration parameters (see node configuration) -
Provide automatic testing (see the testing guide). -
For new features and bug fixes, add an item in the appropriate changelog ( docs/protocols/alpha.rst
for the protocol and the environment,CHANGES.rst
at the root of the repository for everything else). -
Select suitable reviewers using the Reviewers
field below. -
Select as Assignee
the next person who should take action on that MR
Edited by Arvid Jakobsson