Fix uses of global state in Workhorse tests

This fixes two cases of Workhorse tests that touch global state in a way where you cannot run the test more than once with -count.

• In the case of S3 session tests, we now reliably reset global state prior to each test
• In the case of queueing tests, we avoid using the global prometheus metrics registry from tests