jobs.py: Reduce FD leaks from queues and process objects

Daniel Silverstonerequested to merge
danielsilverstone-ct/maybe-reduce-fd-leaks into master

Description

As mentioned in #583 (closed) bst can run out of file descriptors in certain circumstances. Garbage collection can take too long to kick in, and thus it's preferable to force cleanup of certain objects as soon as we're done with them.

Changes proposed in this merge request:

  • Only instantiate the multiprocess.Queue in Job.spawn() rather than on Job.__init__()
  • Delete the Queue and Process instances in Job._parent_child_completed()

This merge request, when approved, will close issue/bug: #583 (closed)

Merge request reports