Review use of `shutil.rmtree`

Review uses of rmtree in cases where that might fail

The following discussion from !1638 (merged) should be addressed:

• @traveltissues started a discussion: (+2 comments)

  this will fail if missing appropriate write permissions, is that ok here? will it ever not have permission to remove?