Skip to content

Revert !1164 - Manage GC during pipeline load

James Ennis requested to merge jennis/revert_gc_management into master

Description

This commit reverts the changes from !1164 (merged) - Manage GC during pipeline load.

!1164 (merged) introduced a patch which improved the load time of a project, pre-scheduler. However, this was done knowing that we would have to compromise peak memory usage. See the discussions on !1164 (merged) for more detail.

Here is the benefit it brings to the bst show time of a 6000 element pipeline (with complex dependency relationships):

show

Here is the memory cost:

show-memory

At the time, we must have only benchmarked this patch for the loading of the debian-stack. However, now that we are benchmarking the build (4, 8 and 12 builders) of the base-files subset of the debian stack, we spotted that when this patch landed, our build times (and peak memory usage during build and show) hiked.

Here is the effect on build times:

build-time

Here is the effect on peak memory usage during the build:

build-memory

Now that we can clearly see the effect that this patch has on the overall build time, I do not think a ~3 sec (from a total of ~22 sec) gain in load time is worth a 100M increase in peak memory usage for both the loading and building of the pipeline as well as a ~90s increase (from a total of ~300s) in the build.

Merge request reports