## Problem In https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93137 we started running all :new: backend spec in random order. To maintainer `master` stability, all existing backend specs were excluded (via [`spec/support/rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml)) from running in random order so they still run in "defined" order (`rspec --order defined`). ## Why This Matters Running backend specs in random order helps GitLab by: **🔍 Detecting Hidden Dependencies**: Random order execution reveals test dependencies that aren't immediately obvious when tests run in a predictable sequence. This helps identify and fix brittle tests that rely on specific execution order or shared state. **🚀 Improving Test Reliability**: Tests that pass in random order are more robust and reliable, reducing flaky test failures in CI/CD pipelines. This leads to faster development cycles and fewer false negatives. **🛡️ Preventing Regressions**: Order-independent tests are less likely to break when new tests are added or existing ones are modified, making the test suite more maintainable over time. **⚡ Better Developer Experience**: Developers can run subsets of tests locally without worrying about execution order, making debugging and development more efficient. **📊 Higher Test Quality**: Random order execution enforces better testing practices like proper test isolation, setup/teardown, and avoiding global state mutations. ## Proposed solution To make spec files run in random order, check their order dependency with: ```shell scripts/rspec_check_order_dependence <path> ... scripts/rspec_check_order_dependence spec/models/project_spec.rb scripts/rspec_check_order_dependence spec/models/project_spec.rb spec/models/project_label_spec.rb scripts/rspec_check_order_dependence spec/models/project* ``` If the specs pass the check the script removes them from `rspec_order_todo.yml` automatically. If the specs fail the check they must be fixed before they can run in random order. See https://docs.gitlab.com/ee/development/testing_guide/best_practices.html#test-order ## The list As of 2023-04-19 the list [`spec/support/rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml) contains over 10.000 entries. ## Implementation Guide Any help is much appreciated :heart: 1. Pick entries from [`spec/support/rspec_order_todo.yml`](https://gitlab.com/gitlab-org/gitlab/-/blob/master/spec/support/rspec_order_todo.yml) 2. Check their order dependency via `scripts/rspec_check_order_dependence` 3. If specs - :white_check_mark: Pass, commit the changes in `spec/support/rspec_order_todo.yml` - :x: Fail - :a: Fix the specs and go to step 2 - :b: Create an issue and ping `@splattael` for triage 4. Submit an MR and mention this issue via `Contributes to #407877` 5. :tada: ## Example MRs * https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93143 - no changes needed * https://gitlab.com/gitlab-org/gitlab/-/merge_requests/93187 - a few adjustments needed ## Status <!-- STATUS NOTE START --> ## Status 2025-10-08 :arrow_forward: next 1. Updated MR https://gitlab.com/gitlab-org/gitlab/-/merge_requests/206032+ based on review suggestions, the keep now 1. Excludes EE specs 2. Works with rspec bisect to give minimal reproduction command on failure. 2. [Run the keep weekly in a scheduled pipeline](https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/merge_requests/256) _Copied from https://gitlab.com/groups/gitlab-org/-/epics/19585#note_2805534403_ <!-- STATUS NOTE END -->