Capture scheduler interactions via file
One issue that we didn't encounter in all the testing of the ff_slurm_sacct
configuration was some cases where output from other services corrupted output from the initial shell action (e.g, sacct
after a job has completed). This MR corrects this issue by shifting to dumping the stdout/stderr to a file which is then parsed, ignoring any potential login related output we cannot efficiently parse. All files are removed after the completion of a job through standard cleanup mechanisms.