Attempt to remove only unlisted files from env directory

Closes #3971 (closed)

From a quick check, having stale directories inside envdir to a chpst is harmless.