Skip to content

AST-1564 Ensure start-dask script works for MID pipeline

Maik Nijhuis requested to merge ast-1564-forward-virtualenv into main
  • Forward the current virtual environment to the nodes when running the script. This enables using the script in other virtual environments too. For example, the MID selfcal pipeline.
  • Support forwarding DASK_WORKER_ARGS to dask workers, e.g., containing "--resources subprocess_slots=1" for the MID pipeline.
  • Make the start-dask script a stand-alone script / do not require source-ing it anymore.
  • Add the number of expected workers as argument to wait_for_dask_workers.py.
Edited by Maik Nijhuis

Merge request reports