Skip to content

Batch commands for remote execution

Background

The remote execution currently issues a separate remote action for each Sandbox.run() call. E.g., for BuildElements this means that each configure, build, install and strip command is executed separately. This approach has the following issues:

  • As file timestamps are not preserved in CAS, each command starts execution with the same timestamp for all staged files. Some projects such as cairo fail to build if the timestamps are not preserved between configure and make
  • Executing a command on a remote worker has an overhead. While we aim to keep this overhead as small as possible, the overall build time for an element is longer with the current individual issuing of commands

Task description

As already contemplated in the original Proposal for Remote Execution, a way to solve these issues is to allow plugins to batch commands, i.e., group multiple Sandbox.run() calls together. This requires designing an extension to the public plugin-facing API.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information