Skip to content

Introduce Remote Dev fast_spec_helper

Chad Woolley requested to merge caw-rd-fast-spec-helper into master

What does this MR do and why?

TL;DR

Allows specs which use this to run in ~8 seconds, vs. ~22 seconds with spring, and ~55 seconds without spring.

Description

Introduces a ee/spec/lib/remote_development/fast_spec_helper.rb which can be used by Category:Remote Development domain logic classes under ee/spec/lib/remote_development if they have little or no dependency on the rest of the Rails monolith. This helper makes running single specs during TDD much faster (I tried to link to rspec docs for context, but they are currently down with heroku failure).

It requires the main spec/fast_spec_helper.rb, and just like that one, it includes a guard to exit early and be a no-op if the full spec-helper is already loaded.

The crux of this helper is the following:

RSpec.configure do |config|
  config.include NextInstanceOf
  config.mock_with :rspec do |mocks|
    mocks.verify_doubled_constant_names = false
  end
end

...which allows usage of instance_double with string classes, even if the class is not loaded.

Without this config change, we would not be able to use this helper on most of the remote dev specs, because most of them use model class instances, but since they are injected via the architecture, we can easily mock them with this approach.

Doc updates

Also includes a related update to the Category:Remote Development domain README.

These docs explicitly state that this approach should not be used in classes which create or otherwise heavily interact with ActiveRecord model classes or other classes, in order to provide better test coverage.

Potential Concerns

Test coverage

This change is safe and does not introduce any risk, because in a full suite (i.e. when run via CI or scripts/remote_development/run-smoke-test-suite.sh) it will still use the full spec_helper, in which mocks.verify_doubled_constant_names = true.

Type safety

See the following comment for a detailed explanation on why some of the experimental rightward-assignment type safety with hardcoded references to Monolith classes/constants were removed to facilitate wider usage of fast_spec_helper:

!126785 (comment 1494395384)

Additional changes

This MR was extracted out of the refactor in Remote Development Railway Oriented Programming... (#418738 - closed), which is already a very large MR, so this reduces it somewhat.

This MR also includes some minor unrelated refactor/cleanup, also towards the goal of reducing the size of that MR.

Example of before and after spec run timings

BEFORE (WITH SPRING)

Average ~21 seconds total spec execution time:

gitlab (caw-rd-fast-spec-helper *=)$ time bin/spring rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb
Running via Spring preloader in process 39234
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 8.61987 seconds
........

Finished in 13.22 seconds (files took 5.53 seconds to load)
8 examples, 0 failures

Randomized with seed 36767

[TEST PROF INFO] Time spent in factories: 00:00.015 (0.08% of total time)
bin/spring rspec ee/spec/lib/remote_development/workspaces/create/main_spec.r  0.44s user 0.10s system 2% cpu 22.223 total
gitlab (caw-rd-fast-spec-helper *=)$ time bin/spring rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb
Running via Spring preloader in process 39504
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 7.855889 seconds
........

Finished in 11.73 seconds (files took 5.44 seconds to load)
8 examples, 0 failures

Randomized with seed 36767

[TEST PROF INFO] Time spent in factories: 00:00.014 (0.09% of total time)
bin/spring rspec ee/spec/lib/remote_development/workspaces/create/main_spec.r  0.44s user 0.10s system 2% cpu 19.495 total

BEFORE (WITHOUT SPRING)

Average ~55 seconds total spec execution time:

gitlab (caw-rd-fast-spec-helper *=)$ bin/spring stop
Spring stopped.
gitlab (caw-rd-fast-spec-helper *=)$ time bin/spring rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb
[TEST PROF INFO] Spring detected
Running via Spring preloader in process 40103
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 6.91038 seconds
........

Finished in 12.64 seconds (files took 4.79 seconds to load)
8 examples, 0 failures

Randomized with seed 36893

[TEST PROF INFO] Time spent in factories: 00:00.020 (0.12% of total time)
bin/spring rspec ee/spec/lib/remote_development/workspaces/create/main_spec.r  0.43s user 0.08s system 0% cpu 56.803 total
gitlab (caw-rd-fast-spec-helper *=)$ bin/spring stop
Spring stopped.
gitlab (caw-rd-fast-spec-helper *=)$ time bin/rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 6.426576 seconds
........

Finished in 11.67 seconds (files took 39.96 seconds to load)
8 examples, 0 failures

Randomized with seed 23760

[TEST PROF INFO] Time spent in factories: 00:00.023 (0.14% of total time)
bin/rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb  41.41s user 9.48s system 94% cpu 53.783 total

AFTER

Average ~8 seconds total spec execution time:

gitlab (caw-rd-fast-spec-helper=)$ time bin/rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}
........

Finished in 0.06017 seconds (files took 5.57 seconds to load)
8 examples, 0 failures

Randomized with seed 42422

bin/rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb  5.20s user 2.01s system 90% cpu 7.941 total
gitlab (caw-rd-fast-spec-helper=)$ time bin/rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}
........

Finished in 0.05971 seconds (files took 5.6 seconds to load)
8 examples, 0 failures

Randomized with seed 7662

bin/rspec ee/spec/lib/remote_development/workspaces/create/main_spec.rb  5.18s user 2.04s system 89% cpu 8.038 total

How to set up and validate locally

See section above for example spec runs.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Chad Woolley

Merge request reports