Skip to content

Optimize runner feature tests

What does this MR do and why?

This change makes feature tests faster by removing redundant setup steps in tests.

Result

We gain ~30 seconds in each test suite (local runs on M1 Macbook Pro):

Spec Before After
spec/features/admin/admin_runners_spec.rb
Finished in 6 minutes 36 seconds (files took 12.14 seconds to load)
$ bundle exec rspec spec/features/admin/admin_runners_spec.rb --profile
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 3.032256 seconds
Starting the Capybara driver server...
Capybara starting Puma...
* Version 6.4.0 , codename: The Eagle of Durango
* Min threads: 0, max threads: 4
* Listening on unix:///var/folders/yb/yqb2vdd94kb3kw0xmhfbjqmc0000gn/T/20231208-97180-vlwrdx
.....................................................................

Top 10 slowest examples (85.84 seconds, 21.7% of total time):
  Admin Runners Admin Runners page runners creation shows a create button
    12.05 seconds ./spec/features/admin/admin_runners_spec.rb:30
  Admin Runners Admin Runners page when there are runners filter by tag shows correct runner when tag is selected and search term is entered
    9.5 seconds ./spec/features/admin/admin_runners_spec.rb:453
  Admin Runners Admin Runners page when there are runners filter by type shows correct runner when type is selected and search term is entered
    8.95 seconds ./spec/features/admin/admin_runners_spec.rb:358
  Admin Runners Admin Runners page when there are runners with an instance runner behaves like pauses, resumes and deletes a runner pauses and resumes runner
    8.87 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:111
  Admin Runners Admin Runners page when there are runners filter by type maintains the same filter when switching between runner types
    8.73 seconds ./spec/features/admin/admin_runners_spec.rb:378
  Admin Runners Admin Runners page when there are runners filter by status shows correct runner when status is selected and search term is entered
    7.68 seconds ./spec/features/admin/admin_runners_spec.rb:271
  Admin Runners Admin Runners page when there are runners filter by tag behaves like filters by tag shows correct runner when tag matches
    7.63 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:178
  Admin Runners Admin Runners page when there are runners with multiple runners behaves like deletes runners in bulk when selecting all for deletion behaves like shows no runners registered shows 0 count and the empty state
    7.56 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:66
  Admin Runners Admin Runners page when there are runners filter by tag when tag does not match behaves like shows no runners found shows "no runners" message
    7.48 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:80
  Admin Runners Admin Runners page when there are runners filter by tag when tag does not match shows no runner
    7.39 seconds ./spec/features/admin/admin_runners_spec.rb:448

Finished in 6 minutes 36 seconds (files took 12.14 seconds to load)
69 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:02.325 (0.58% of total time)
Finished in 6 minutes 3 seconds (files took 12.01 seconds to load)
$ bundle exec rspec spec/features/admin/admin_runners_spec.rb --profile
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 3.402788 seconds
Starting the Capybara driver server...
Capybara starting Puma...
* Version 6.4.0 , codename: The Eagle of Durango
* Min threads: 0, max threads: 4
* Listening on unix:///var/folders/yb/yqb2vdd94kb3kw0xmhfbjqmc0000gn/T/20231211-88129-2xok34
.............................................................

Top 10 slowest examples (85.79 seconds, 23.6% of total time):
  Admin Runners Runner show page shows runner details
    11.51 seconds ./spec/features/admin/admin_runners_spec.rb:548
  Admin Runners Admin Runners page when there are runners with an instance runner behaves like pauses, resumes and deletes a runner pauses and resumes runner
    9.44 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:82
  Admin Runners Admin Runners page when there are runners filter by tag shows correct runner when tag is selected and search term is entered
    9.18 seconds ./spec/features/admin/admin_runners_spec.rb:434
  Admin Runners Admin Runners page when there are runners filter by type shows correct runner when type is selected and search term is entered
    8.59 seconds ./spec/features/admin/admin_runners_spec.rb:346
  Admin Runners Admin Runners page when there are runners with multiple runners behaves like deletes runners in bulk when selecting all for deletion behaves like shows no runners registered shows 0 count and the empty state
    8.36 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:40
  Admin Runners Admin Runners page when there are runners filter by type maintains the same filter when switching between runner types
    8.31 seconds ./spec/features/admin/admin_runners_spec.rb:366
  Admin Runners Admin Runners page when there are runners with an instance runner behaves like pauses, resumes and deletes a runner deletes runner confirms runner deletion
    7.75 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:105
  Admin Runners Admin Runners page when there are runners filter by tag when tag does not match behaves like shows no runners found shows "no runners" message
    7.63 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:54
  Admin Runners Admin Runners page when there are runners filter by tag behaves like filters by tag shows correct runner when tag matches
    7.57 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:143
  Admin Runners Admin Runners page when there are runners filter by status shows correct runner when status is selected and search term is entered
    7.45 seconds ./spec/features/admin/admin_runners_spec.rb:259

Finished in 6 minutes 3 seconds (files took 12.01 seconds to load)
61 examples, 0 failures

Randomized with seed 42636

[TEST PROF INFO] Time spent in factories: 00:02.496 (0.68% of total time)
spec/features/groups/group_runners_spec.rb
Finished in 2 minutes 37.6 seconds (files took 12.01 seconds to load)
$ bundle exec rspec spec/features/groups/group_runners_spec.rb --profile
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 2.285289 seconds
Starting the Capybara driver server...
Capybara starting Puma...
* Version 6.4.0 , codename: The Eagle of Durango
* Min threads: 0, max threads: 4
* Listening on unix:///var/folders/yb/yqb2vdd94kb3kw0xmhfbjqmc0000gn/T/20231208-97977-kz7cuo
.........................................

Top 10 slowest examples (57.27 seconds, 36.3% of total time):
  Group Runners Group runners page when logged in as group maintainer with no runners shows tabs with total counts equal to 0
    8.02 seconds ./spec/features/groups/group_runners_spec.rb:32
  Group Runners Group runners page when logged in as group owner with an online project runner behaves like pauses, resumes and deletes a runner pauses and resumes runner
    7.07 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:111
  Group Runners Group runners page when logged in as group owner with an online group runner behaves like pauses, resumes and deletes a runner pauses and resumes runner
    6.9 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:111
  Group Runners Group runners page when logged in as group owner with multiple runners behaves like deletes runners in bulk when selecting all for deletion behaves like shows no runners registered shows 0 count and the empty state
    5.54 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:66
  Group Runners Group runners page when logged in as group owner with an online group runner behaves like pauses, resumes and deletes a runner deletes runner deletes a runner
    5.31 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:139
  Group Runners Group runners page when logged in as group owner with an online project runner behaves like pauses, resumes and deletes a runner deletes runner deletes a runner
    5.23 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:139
  Group Runners Group runners page when logged in as group maintainer with an online instance runner when selecting 'Show only inherited' behaves like shows runner details from list shows runner details page
    5.16 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:98
  Group Runners Group runners page when logged in as group maintainer filter by tag behaves like filters by tag shows correct runner when tag matches
    4.94 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:178
  Group Runners Group runners page when logged in as group maintainer with an online group runner behaves like shows runner details from list shows runner details page
    4.62 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:98
  Group Runners Group runners page when logged in as group maintainer with an online group runner when description does not match shows no runner
    4.48 seconds ./spec/features/groups/group_runners_spec.rb:70

Finished in 2 minutes 37.6 seconds (files took 12.01 seconds to load)
41 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:01.926 (1.21% of total time)
Finished in 1 minute 55.87 seconds (files took 12.38 seconds to load)
$ bundle exec rspec spec/features/groups/group_runners_spec.rb --profile
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 3.162079 seconds
Starting the Capybara driver server...
Capybara starting Puma...
* Version 6.4.0 , codename: The Eagle of Durango
* Min threads: 0, max threads: 4
* Listening on unix:///var/folders/yb/yqb2vdd94kb3kw0xmhfbjqmc0000gn/T/20231211-88989-pxjwpo
.............................

Top 10 slowest examples (54.13 seconds, 46.7% of total time):
  Group Runners Group runner show page when logged in as group maintainer user views runner details
    7.28 seconds ./spec/features/groups/group_runners_spec.rb:243
  Group Runners Group runners page when logged in as group owner with an online project runner behaves like pauses, resumes and deletes a runner pauses and resumes runner
    6.6 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:82
  Group Runners Group runners page when logged in as group owner with an online group runner behaves like pauses, resumes and deletes a runner pauses and resumes runner
    6.46 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:82
  Group Runners Group runners page when logged in as group owner with multiple runners behaves like deletes runners in bulk when selecting all for deletion behaves like shows no runners registered shows 0 count and the empty state
    5.26 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:40
  Group Runners Group runners page when logged in as group owner with an online project runner behaves like pauses, resumes and deletes a runner deletes runner confirms runner deletion
    5.18 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:105
  Group Runners Group runners page when logged in as group owner with an online group runner behaves like pauses, resumes and deletes a runner deletes runner confirms runner deletion
    5.1 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:105
  Group Runners Group runners page when logged in as group maintainer filter by tag behaves like filters by tag shows correct runner when tag matches
    4.75 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:143
  Group Runners Group runners page when logged in as group maintainer with an online instance runner when selecting 'Show only inherited' behaves like shows runner summary and navigates to details shows runner summary and details
    4.73 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:60
  Group Runners Group runners page when logged in as group maintainer with an online project runner behaves like shows runner summary and navigates to details shows runner summary and details
    4.4 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:60
  Group Runners Group runners page when logged in as group maintainer with an online group runner when description does not match behaves like shows no runners found shows "no runners" message
    4.37 seconds ./spec/support/shared_examples/features/runners_shared_examples.rb:54

Finished in 1 minute 55.87 seconds (files took 12.38 seconds to load)
29 examples, 0 failures

Randomized with seed 43656

[TEST PROF INFO] Time spent in factories: 00:01.898 (1.61% of total time)

Issues

After this is merged, I propose to close #420824 (closed) and #421743 (closed), and revisit these slow tests if the issues get reopened by the test bot.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #421743 (closed)

Edited by Miguel Rincon

Merge request reports