Add CI job to validate newly added specs can use fast_spec_helper

Summary

Create a CI job that automatically validates whether newly added spec files using require 'spec_helper' can actually run with require 'fast_spec_helper' instead. This will help prevent developers from unnecessarily using the slower spec_helper when fast_spec_helper would suffice.

Background

As part of the ongoing effort to improve test performance (related to #581686 (closed)), we're converting specs to use fast_spec_helper where possible. However, new specs are still being added with spec_helper when they could use fast_spec_helper.

This issue proposes adding a CI job that will:

  1. Detect newly added spec files that use require 'spec_helper'
  2. Temporarily replace spec_helper with fast_spec_helper in those files
  3. Run the specs to see if they pass with fast_spec_helper
  4. Fail the CI job if the specs pass with fast_spec_helper, indicating the developer should update their spec to use fast_spec_helper instead

Proposed Implementation

The CI job should be similar to existing validation jobs like the workspaces domain smoke test script.

The job should:

  • Run on merge request pipelines
  • Identify newly added spec files (files added in the current branch compared to the target branch)
  • Filter for spec files containing require 'spec_helper'
  • For each file:
    • Create a temporary version with fast_spec_helper instead
    • Run the spec in isolation (to avoid false positives from shared dependencies)
    • Track which specs pass with fast_spec_helper
  • Exit with failure if any specs can run with fast_spec_helper, providing clear feedback to the developer

Benefits

  • Prevents regression: ensures new specs use the fastest helper possible
  • Educates developers: provides immediate feedback during code review
  • Improves test performance: reduces unnecessary Rails environment loading
  • Automates manual review: removes the need for reviewers to manually check this

Related

Edited by 🤖 GitLab Bot 🤖