Batch commands for remote execution
Background
The remote execution currently issues a separate remote action for each Sandbox.run()
call. E.g., for BuildElements this means that each configure, build, install and strip command is executed separately. This approach has the following issues:
- As file timestamps are not preserved in CAS, each command starts execution with the same timestamp for all staged files. Some projects such as cairo fail to build if the timestamps are not preserved between
configure
andmake
- Executing a command on a remote worker has an overhead. While we aim to keep this overhead as small as possible, the overall build time for an element is longer with the current individual issuing of commands
Task description
As already contemplated in the original Proposal for Remote Execution, a way to solve these issues is to allow plugins to batch commands, i.e., group multiple Sandbox.run()
calls together. This requires designing an extension to the public plugin-facing API.