Skip to content
Snippets Groups Projects

Run all new backend specs in random order

Merged Peter Leitzen requested to merge pl-rspec-order into master
Files
5
@@ -83,6 +83,27 @@ You can silence deprecation warnings by setting the environment variable
SILENCE_DEPRECATIONS=1 bin/rspec spec/models/project_spec.rb
```
### Test order
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93137) in GitLab 15.3.
All _new_ specs are run in [random order](https://gitlab.com/gitlab-org/gitlab/-/issues/337399)
to surface flaky tests that are dependent test order.
For a list of spec files which are still run in defined order, see [`rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml).
To make specs run in random order:
1. Check if specs fail when run in random order:
```shell
scripts/rspec_check_order_dependence spec/models/project_spec.rb
```
1. Fix the spec failures if needed
1. Remove checked spec files from [`rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml).
### Test speed
GitLab has a massive test suite that, without [parallelization](../pipelines.md#test-suite-parallelization), can take hours
Loading