Detect specs that can use fast_spec_helper and update them

We have many spec files which could be using fast_spec_helper instead of spec_helper, allowing for much faster (sub-second) spec execution and feedback when running on a local machine, compared to ~7 seconds for spec_helper. The problem is that we rely on a developer coming across such files, understanding that they don't need to load spec_helper, and creating a MR to fix it.

Proposal

I asked Duo to create a script to automate the process:

Write a ruby script that will go through each of our spec files that is using spec_helper, change it to use fast_spec_helper, test it with bundle exec rspec and revert the change if the test fails, and keeping the change if it succeeds.

Script

@@ -0,0 +1,112 @@
#!/usr/bin/env ruby
# frozen_string_literal: true

require 'fileutils'
require 'open3'

class SpecHelperConverter
  SPEC_HELPER_PATTERN = /^require ['"]spec_helper['"]/
  FAST_SPEC_HELPER = "require 'fast_spec_helper'"
  
  attr_reader :stats

  def initialize
    @stats = {
      total: 0,
      converted: 0,
      failed: 0,
      skipped: 0
    }
  end

  def run
    spec_files = find_spec_files_with_spec_helper
    
    puts "Found #{spec_files.size} spec files using spec_helper"
    puts "Starting conversion process...\n\n"

    spec_files.each_with_index do |file, index|
      @stats[:total] += 1
      puts "[#{index + 1}/#{spec_files.size}] Processing: #{file}"
      
      process_file(file)
      puts ""
    end

    print_summary
  end

  private

  def find_spec_files_with_spec_helper
    spec_files = Dir.glob('spec/**/*_spec.rb')
    
    spec_files.select do |file|
      content = File.read(file)
      content.match?(SPEC_HELPER_PATTERN)
    end
  end

  def process_file(file_path)
    original_content = File.read(file_path)
    
    # Replace spec_helper with fast_spec_helper
    modified_content = original_content.sub(SPEC_HELPER_PATTERN, FAST_SPEC_HELPER)
    
    if modified_content == original_content
      puts "  ⊘ Skipped: No spec_helper require found"
      @stats[:skipped] += 1
      return
    end

    # Write the modified content
    File.write(file_path, modified_content)
    puts "  ✎ Changed spec_helper to fast_spec_helper"

    # Run the spec
    if run_spec(file_path)
      puts "  ✓ Tests passed - keeping change"
      @stats[:converted] += 1
    else
      puts "  ✗ Tests failed - reverting change"
      File.write(file_path, original_content)
      @stats[:failed] += 1
    end
  end

  def run_spec(file_path)
    puts "  ⚙ Running: bundle exec rspec #{file_path}"
    
    stdout, stderr, status = Open3.capture3("bundle exec rspec #{file_path}")
    
    if status.success?
      true
    else
      puts "  Error output:"
      stderr.lines.first(5).each { |line| puts "    #{line}" }
      false
    end
  end

  def print_summary
    puts "=" * 60
    puts "CONVERSION SUMMARY"
    puts "=" * 60
    puts "Total files processed:    #{@stats[:total]}"
    puts "Successfully converted:   #{@stats[:converted]} ✓"
    puts "Failed (reverted):        #{@stats[:failed]} ✗"
    puts "Skipped:                  #{@stats[:skipped]} ⊘"
    puts "=" * 60
    
    if @stats[:converted] > 0
      success_rate = (@stats[:converted].to_f / @stats[:total] * 100).round(2)
      puts "\nSuccess rate: #{success_rate}%"
    end
  end
end

# Run the converter
if __FILE__ == $PROGRAM_NAME
  converter = SpecHelperConverter.new
  converter.run
end

Executing this inside the GDK console will modify all spec files that can leverage fast_spec_helper:

> require_relative 'scripts/convert_to_fast_spec_helper'
> SpecHelperConverter.new.run
Found 10417 spec files using spec_helper
Starting conversion process...

[1/10417] Processing: spec/bin/feature_flag_spec.rb
  ✎ Changed spec_helper to fast_spec_helper
  ⚙ Running: bundle exec rspec spec/bin/feature_flag_spec.rb
  Error output:
    warning: parser/current is loading parser/ruby33, which recognizes 3.3.10-compliant syntax, but you are running 3.3.9.
    Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
    /Users/pedropombeiro/gitlab-development-kit/gitlab/lib/feature.rb:7: warning: already initialized constant Feature::SUPPORTED_MODELS
    /Users/pedropombeiro/gitlab-development-kit/gitlab/lib/feature.rb:7: warning: previous definition of SUPPORTED_MODELS was here
  ✗ Tests failed - reverting change

[2/10417] Processing: spec/cells/claims/organizations/organization_spec.rb
  ✎ Changed spec_helper to fast_spec_helper
  ⚙ Running: bundle exec rspec spec/cells/claims/organizations/organization_spec.rb
  Error output:
    warning: parser/current is loading parser/ruby33, which recognizes 3.3.10-compliant syntax, but you are running 3.3.9.
    Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
  ✗ Tests failed - reverting change