Builds can fail depending on host resources, some control is desireable

In some scenarios, builds can fail depending on how much resources they have available. This is worsened when coupled with fault intolerant tooling, such as compilers which abort at the first sight of an OOM condition.

A recently observed case is when building LLVM (which manages to link many large binaries in parallel). In this case the build was working fine with a reasonable amount of available RAM, but begins to fail when given more CPUs, triggering more parallel linking and reaching the OOM condition.

Various approaches are possible to address the load here. A declarative approach might include some additional user configuration to describe the capabilities of the host, and / or some extension to the .bst format to allow one to describe the resource consumption characteristics of a given build (for instance, one might be able to express the nature of the LLVM build in a llvm.bst file so as to inform BuildStream about the per-job memory requirements of the element).

None of the approaches I've come up with so far are in any way attractive, though; open to suggestions.