feat: configurable docker build parallelism
Description
Docker buildkit will parallelise build stage that don't depend on each other, which typically speeds up the build, but when running a highly parallel job on low-spec hardware, it can also have negative effects, in extreme cases completely blocking the build.
The MFE build is highly parallel - without limiting the parallelism, docker would try to build all MFEs at the same time, which caused a lot of issues on our CI workers.
Limiting the parallelism to 2 seems to fix the problem.
Testing instructions
Check out these two pipelines which both succeeded in a decent amount of time:
- https://gitlab.com/opencraft/ops/grove-stage-digitalocean/-/pipelines/843091870
- https://gitlab.com/opencraft/ops/grove-stage-digitalocean/-/pipelines/843208699
Note: With this change the MFE build step is quite stable, but I'm still getting random failures on the deploy step. It usually succeeds after a retry or two. Not sure if that's related to Olive or whether it's a general issue.
Checklist
If any of the items below is not applicable, do not remove them, but put a check in it.
-
All providers include the new feature/change -
All affected providers can provision new clusters -
Unit tests are added/updated -
Documentation is added/updated -
The TOOLS_CONTAINER_IMAGE_VERSION
in ci_vars.yml is updated -
The grove-template repository is updated