Further parallelize builds
Why is this change being made?
This change is being made to reduce overall pipeline times. By further parallelizing the build to 9 partial builds, we can get each section down to 7 +/- minutes, thus shaving a few more minutes off our overall pipeline times.
The 9 partials are now defined as:
-
IMAGES: We have a lot of images. So we let middleman copy them. Maybe we can de-middleman it in the future -
BLOG_POST_OLD: All blog posts up to 2017 -
BLOG_POST_NEW: All blog posts since 2018 -
RELEASE_BLOG: Release blog posts -
HANDBOOK_ENGINEERING: The engineering handbook -
HANDBOOK_OTHER: Other handbooks -
ASSETS: javascripts, stylesheets, icons, and pdfs -
COMPANY: All company pages and team images -
ALL_OTHERS: All pages and files which do not fit into the categories above
This is listed as a smaller scope improvement to be made in the "monorepo approval" issue.
Why not devide it further? Because of the DAG limit
We allow the review and deploy jobs to start immediately after build_branch and build_proxy_resources are complete. We recently increased the DAG limit from 5 to 10 to allow for this. With the build_branch job now divided into 9 parallels, we've hit the DAG limit of 10.
Does this MR meet the acceptance criteria?
Conformity
-
Added description to this MR explaining the reasons for the proposed change, per say-why-not-just-what
Related #1265 (closed)
Edited by Chad Woolley