Skip to content

Parallelize integration script

Dmitry Gruzd requested to merge parallelize-integration-script into main

What does this MR do and why?

This MR reduces the execution time of the integration script (spec/scripts/integration.sh) execution by 3.3x.

Version Runtime (m:ss)
Original implementation 3:17
Secuential mode in this MR 2:13
Parallel mode in this MR 1:00

Screenshots or screen recordings

The original script before these changes:

$ git checkout main
Switched to branch 'main'
Your branch is up to date with 'origin/main'.
$ time ./spec/scripts/integration.sh
...
./spec/scripts/integration.sh  6.05s user 2.20s system 4% cpu 3:17.82 total

Sequential mode in this branch:

$ time ./spec/scripts/integration.sh
integration.sh: Using sequential mode [default]
...
Elapsed Time: 134 seconds
./spec/scripts/integration.sh  4.07s user 1.56s system 4% cpu 2:13.61 total

Parallel mode in this branch:

$ time PARALLEL=1 ./spec/scripts/integration.sh
integration.sh: Using parallel mode
...
Elapsed Time: 60 seconds
PARALLEL=1 ./spec/scripts/integration.sh  3.93s user 1.78s system 9% cpu 1:00.60 total
Click to expand

parallel_integration.svg

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Edited by Dmitry Gruzd

Merge request reports