Improve automatic retries of failing specs
We have seen a number of transient failures over the last few months that disappear if you run the specs in isolation (e.g. rspec spec/foo_spec.rb). Could we make our testing infrastructure more resilient by doing something as the following?
- If a spec fails, flag the file as failed
- At the end of the other specs, re-run the specs with the failed group of specs