Consider using `parallel_tests` to speed up the test suite on the CI
As suggested by @engwan in https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/12688#note_171255576:
Another idea I had was to use something like https://github.com/grosser/parallel_tests to parallelize our tests within a job so we could use a smaller number of jobs. There are docs on how to use this with knapsack: https://github.com/KnapsackPro/knapsack_pro-ruby#how-to-run-knapsack_pro-with-parallel_tests-gem
Rails 5.1.5 introduced parallel tests but we can't use that with RSpec so we'd need to use that gem.
This makes us fully utilize our runners and potentially save a lot of machine time.
If we have dual-core runners, we could theoretically cut our machine time by almost 50% by splitting to only 25 jobs and keep the running time almost the same.
That is huge and I think is worth an experiment to see how hard it is to get something like that setup.
Some things we need to consider:
- Separate DB per process (https://github.com/grosser/parallel_tests#add-to-configdatabaseyml)
- Separate Redis DB per process
- Separate Gitaly process
- Separate
tmp/tests
directory - Separate Capybara server (https://github.com/grosser/parallel_tests/wiki#with-capybara040selenium----by-rgo)