This should do a better job of removing the test jobs, that have been
cluttering up our UI.

This also has the added benefit of allowing us to retry these, which
wasn't working before.