Raise the panic anew instead of just printing its value

This enables traces to still be printed after the cleanup has been done.

Maybe you want to add a setting to control the cleanup (to avoid cleaning up when we want to examine the dirs).