Skip to content
Snippets Groups Projects

Run all new backend specs in random order

Merged Peter Leitzen requested to merge pl-rspec-order into master
1 file
+ 20
0
Compare changes
  • Side-by-side
  • Inline
@@ -83,6 +83,26 @@ You can silence deprecation warnings by setting the environment variable
SILENCE_DEPRECATIONS=1 bin/rspec spec/models/project_spec.rb
```
### Test order
> [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93137) in GitLab 15.3.
All _new_ spec files are run in [random order](https://gitlab.com/gitlab-org/gitlab/-/issues/337399)
to surface flaky tests that are dependent test order.
For a list of spec files which are still run in defined order, see [`rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml).
To make spec files run in random order, check their order dependency with:
```shell
scripts/rspec_check_order_dependence spec/models/project_spec.rb
```
If they pass the check the spec files the script removes them from
[`rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml).
If they fail the check spec failures must be fixed.
### Test speed
GitLab has a massive test suite that, without [parallelization](../pipelines.md#test-suite-parallelization), can take hours
@@ -422,7+442,7 @@
### System / Feature tests
NOTE:
Before writing a new system test, [please consider **not**
writing one](testing_levels.md#consider-not-writing-a-system-test)!
- Feature specs should be named `ROLE_ACTION_spec.rb`, such as
Loading