We were only shutting it down when the queue size was non-empty. Outdent some code so it always happens. Fixes #1794.