Skip to content

[RSpec random order] Processed 20 specs: 1 passed, 19 failed

Project token used in gitlab-org/quality/engineering-productivity/team/-/settings/ci_cd as `HOUSEKEEPER_GITLAB_API_TOKEN`requested to merge
rspec-order-checker-batch_20251020_5505 into master

RSpec Order Dependency Check Results

Processed 20 spec files from spec/support/rspec_order_todo.yml:

  • 1 specs passed and can now run in random order
  • 19 specs failed and were moved to spec/support/rspec_order_failures.yml

Passed specs (1):

  • spec/config/inject_enterprise_edition_module_spec.rb

Failed specs (19):

  • spec/bin/feature_flag_spec.rb
  • spec/bin/sidekiq_cluster_spec.rb
  • spec/channels/application_cable/connection_spec.rb
  • spec/commands/metrics_server/metrics_server_spec.rb
  • spec/commands/sidekiq_cluster/cli_spec.rb
  • spec/config/application_spec.rb
  • spec/config/mail_room_spec.rb
  • spec/config/object_store_settings_spec.rb
  • spec/config/settings_spec.rb
  • spec/config/smime_signature_settings_spec.rb
  • spec/controllers/acme_challenges_controller_spec.rb
  • spec/controllers/admin/application_settings/appearances_controller_spec.rb
  • spec/controllers/admin/application_settings_controller_spec.rb
  • spec/controllers/admin/applications_controller_spec.rb
  • spec/controllers/admin/ci/variables_controller_spec.rb
  • spec/controllers/admin/clusters_controller_spec.rb
  • spec/controllers/admin/cohorts_controller_spec.rb
  • spec/controllers/admin/dashboard_controller_spec.rb
  • spec/controllers/admin/dev_ops_report_controller_spec.rb

Progress Summary:

  • Processed this batch: 20 specs
  • Passed (can run randomly): 1 specs
  • Failed (moved to failure list): 19 specs
  • Remaining in TODO: 5485 specs (excluding ee specs)
  • TODO list cleanup: Removed 20 processed entries

Debugging Failed Specs:

For specs that failed the order dependency check:

  1. Use the existing script: scripts/rspec_check_order_dependence <spec_file>
  2. The script automatically runs: defined order → reverse order → random order → bisect on failure
  3. Fix the root cause: Remove shared state or add proper cleanup
  4. Re-test: Run the spec again to verify the fix

Relates to #576789

The order dependency checks were performed using scripts/rspec_check_order_dependence which includes comprehensive testing (defined/reverse/random order) and automatic bisect analysis on failures.

This change was generated by gitlab-housekeeper in CI using the Keeps::RspecOrderChecker keep.

To provide feedback on your experience with gitlab-housekeeper please create an issue with the label GitLab Housekeeper and consider pinging the author of this keep.

Merge request reports