Skip to content

feat: configurable docker build parallelism

Matjaz Gregoric requested to merge mtyaka/SE-5939-docker-limits into main

Description

Docker buildkit will parallelise build stage that don't depend on each other, which typically speeds up the build, but when running a highly parallel job on low-spec hardware, it can also have negative effects, in extreme cases completely blocking the build.

The MFE build is highly parallel - without limiting the parallelism, docker would try to build all MFEs at the same time, which caused a lot of issues on our CI workers.

Limiting the parallelism to 2 seems to fix the problem.

Testing instructions

Check out these two pipelines which both succeeded in a decent amount of time:

Note: With this change the MFE build step is quite stable, but I'm still getting random failures on the deploy step. It usually succeeds after a retry or two. Not sure if that's related to Olive or whether it's a general issue.

Checklist

If any of the items below is not applicable, do not remove them, but put a check in it.

  • All providers include the new feature/change
  • All affected providers can provision new clusters
  • Unit tests are added/updated
  • Documentation is added/updated
  • The TOOLS_CONTAINER_IMAGE_VERSION in ci_vars.yml is updated
  • The grove-template repository is updated

Merge request reports