Add new keep to check spec order

Context

Run new backend specs in random order (&19585)

What does this MR do and why?

Adds a housekeeper script keeps/rspec_order_checker.rb to port backend specs in rspec_order_todo.yml from running in specified order to running in random order.

• Processed specs are removed from rspec_order_todo.yml file.
  • If the script is successful for a spec, then no further steps are needed. The spec file will now run in random order.
  • If the script fails, failed specs are moved to rspec_order_failures.yml file (these are still run in order). This ensures that the todo file is only used for tracking specs that are yet to be verified.
• Add tests for the keep

Results

MR [RSpec random order] Processed 1 specs: all fai... (!206239 - closed) created by running the Housekeep using GDK locally.

Steps to reproduce locally

House keep runs automatically on CI based on scheduled pipelines but to simplify testing and not trigger other housekeeping tasks, I used GDK to test this.

• Ensure your GDK setup is up-to-date, and checkout to the gitlab folder inside gdk

  cd <gdk_path>/gitlab

• Checkout to this branch, rebase with master (to ensure there are no changes to rspec_order_todo.yml). The keep will fail if there are any conflicts

• Run the following script in dry mode (remove the -d flag if you want create an MR and view the changes)

bundle exec gitlab-housekeeper -k Keeps::RspecOrderChecker -d -b $(git rev-parse --abbrev-ref HEAD)

NOTE: by default keeps use master branch to compare against change

NOTE: Right now, the script process only 1 file to keep things simple and easily test the end-to-end flow. It can be modified by setting the LIMIT_SPECS constant in the keep file (be sure to commit locally) to test more specs at the same time.

Further steps/enhancements

• Validate runs on CI (add to EP space)
• Using test runtime to group more specs in a single MR
• Support creating multiple MRs when you pass the flag -m <count>

AI Usage

What worked

• Initial code and code organisation
• Generating MR descriptions
• Context gathering related to Housekeeper

What didn't

• Duo struggled with debugging related to rspec bisect(stuck process because of fork issues on mac), housekeep failure because of conflicts with master
  • Lots of incorrect assumptions and running in circles.
• Vibe code
• Chat dying with errors halfway through a session (not necessarily because of context limit)

Overall code accuracy in between 60-70% and I had to clean up vibe code

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Relates to #407877 (closed)

Relates to &19585