Run `ee/spec/services/vulnerabilit*` specs in a random order (!147049) · Merge requests · GitLab.org / GitLab

Michael Becker requested to merge task/run-vulnerability-tasks-in-random-order into master Mar 11, 2024

What does this MR do and why?

While working on a separate issue, I noticed all of these specs are in rspec_order_todo.yml

I ran the following script that indicated it is safe to run these specs in a random order:

for file in ee/spec/services/vulnerabilit* ;
do
    scripts/rspec_check_order_dependence "$file";
done

Output

+ export RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ bin/rspec --order defined --format progress ee/spec/services/vulnerabilities
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}
==> GitLab Workhorse set up in 8.955855658 seconds...

Test environment set up in 10.294566414 seconds
....................................................................................................................................................................................................................................................................................................................

Finished in 10 minutes 14 seconds (files took 36.16 seconds to load)
308 examples, 0 failures

Randomized with seed 45264

[TEST PROF INFO] Time spent in factories: 08:55.482 (83.78% of total time)
+ RSPEC_ORDER=reverse
+ bin/rspec --format progress ee/spec/services/vulnerabilities
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.278207868 seconds
....................................................................................................................................................................................................................................................................................................................

Finished in 7 minutes 31 seconds (files took 13.61 seconds to load)
308 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 06:38.899 (87.98% of total time)
+ bin/rspec --order random --format progress ee/spec/services/vulnerabilities
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 46194

Test environment set up in 1.363234043 seconds
....................................................................................................................................................................................................................................................................................................................

Finished in 8 minutes 29 seconds (files took 11.57 seconds to load)
308 examples, 0 failures

Randomized with seed 46194

[TEST PROF INFO] Time spent in factories: 07:28.226 (87.74% of total time)
+ set +xe


The files passed all checks!

They are likely not order-dependent and can be run in random order and thus
are being removed from 'spec/support/rspec_order_todo.yml':

  * Removing 'ee/spec/services/vulnerabilities'

+ export RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ bin/rspec --order defined --format progress ee/spec/services/vulnerability_exports
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.101684564 seconds
.........................

Finished in 29.65 seconds (files took 11.84 seconds to load)
25 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:24.465 (77.89% of total time)
+ RSPEC_ORDER=reverse
+ bin/rspec --format progress ee/spec/services/vulnerability_exports
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.366100347 seconds
.........................

Finished in 36.68 seconds (files took 13.62 seconds to load)
25 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:29.787 (76.79% of total time)
+ bin/rspec --order random --format progress ee/spec/services/vulnerability_exports
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 55241

Test environment set up in 1.356115575 seconds
.........................

Finished in 33.55 seconds (files took 13.57 seconds to load)
25 examples, 0 failures

Randomized with seed 55241

[TEST PROF INFO] Time spent in factories: 00:27.515 (77.3% of total time)
+ set +xe


The files passed all checks!

They are likely not order-dependent and can be run in random order and thus
are being removed from 'spec/support/rspec_order_todo.yml':

  * Removing 'ee/spec/services/vulnerability_exports'

+ export RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ bin/rspec --order defined --format progress ee/spec/services/vulnerability_external_issue_links
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.280331421 seconds
.................

Finished in 37.52 seconds (files took 14.39 seconds to load)
17 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:31.064 (78.81% of total time)
+ RSPEC_ORDER=reverse
+ bin/rspec --format progress ee/spec/services/vulnerability_external_issue_links
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.304284282 seconds
.................

Finished in 39.14 seconds (files took 12.73 seconds to load)
17 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:32.254 (78.93% of total time)
+ bin/rspec --order random --format progress ee/spec/services/vulnerability_external_issue_links
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 59320

Test environment set up in 1.362348655 seconds
.................

Finished in 37.71 seconds (files took 14.94 seconds to load)
17 examples, 0 failures

Randomized with seed 59320

[TEST PROF INFO] Time spent in factories: 00:30.370 (76.59% of total time)
+ set +xe


The files passed all checks!

They are likely not order-dependent and can be run in random order and thus
are being removed from 'spec/support/rspec_order_todo.yml':

  * Removing 'ee/spec/services/vulnerability_external_issue_links'

+ export RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ bin/rspec --order defined --format progress ee/spec/services/vulnerability_feedback
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.566721594 seconds
........................................................

Finished in 1 minute 29.51 seconds (files took 12.46 seconds to load)
56 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 01:01.363 (66.95% of total time)
+ RSPEC_ORDER=reverse
+ bin/rspec --format progress ee/spec/services/vulnerability_feedback
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.544871522 seconds
........................................................

Finished in 1 minute 23.6 seconds (files took 12.09 seconds to load)
56 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:57.127 (66.84% of total time)
+ bin/rspec --order random --format progress ee/spec/services/vulnerability_feedback
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 14756

Test environment set up in 1.392882214 seconds
........................................................

Finished in 1 minute 37.92 seconds (files took 14.78 seconds to load)
56 examples, 0 failures

Randomized with seed 14756

[TEST PROF INFO] Time spent in factories: 01:07.198 (67.28% of total time)
+ set +xe


The files passed all checks!

They are likely not order-dependent and can be run in random order and thus
are being removed from 'spec/support/rspec_order_todo.yml':

  * Removing 'ee/spec/services/vulnerability_feedback'

+ export RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ bin/rspec --order defined --format progress ee/spec/services/vulnerability_issue_links
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.338738929 seconds
...................................

Finished in 33.93 seconds (files took 13.56 seconds to load)
35 examples, 0 failures

Randomized with seed 4973

[TEST PROF INFO] Time spent in factories: 00:25.143 (69.19% of total time)
+ RSPEC_ORDER=reverse
+ bin/rspec --format progress ee/spec/services/vulnerability_issue_links
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.351974407 seconds
...................................

Finished in 35.61 seconds (files took 13.27 seconds to load)
35 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:27.511 (73.28% of total time)
+ bin/rspec --order random --format progress ee/spec/services/vulnerability_issue_links
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 48351

Test environment set up in 1.342351842 seconds
...................................

Finished in 37.68 seconds (files took 13.11 seconds to load)
35 examples, 0 failures

Randomized with seed 48351

[TEST PROF INFO] Time spent in factories: 00:28.073 (70.6% of total time)
+ set +xe


The files passed all checks!

They are likely not order-dependent and can be run in random order and thus
are being removed from 'spec/support/rspec_order_todo.yml':

  * Removing 'ee/spec/services/vulnerability_issue_links'

Could not find 'ee/spec/services/vulnerability_merge_request_links' in './spec/support/rspec_order_todo.yml'
Aborting...
+ export RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ RSPEC_WARN_MISSING_FEATURE_CATEGORY=0
+ bin/rspec --order defined --format progress ee/spec/services/vulnerability_scanners
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.355479061 seconds
..

Finished in 6.24 seconds (files took 12.48 seconds to load)
2 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:01.660 (19.08% of total time)
+ RSPEC_ORDER=reverse
+ bin/rspec --format progress ee/spec/services/vulnerability_scanners
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 1.37194578 seconds
..

Finished in 6.11 seconds (files took 12.55 seconds to load)
2 examples, 0 failures

[TEST PROF INFO] Time spent in factories: 00:01.713 (20.35% of total time)
+ bin/rspec --order random --format progress ee/spec/services/vulnerability_scanners
warning: parser/current is loading parser/ruby32, which recognizes 3.2.2-compliant syntax, but you are running 3.2.3.
Please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Randomized with seed 51722

Test environment set up in 1.409527595 seconds
..

Finished in 5.91 seconds (files took 14.76 seconds to load)
2 examples, 0 failures

Randomized with seed 51722

[TEST PROF INFO] Time spent in factories: 00:01.702 (20.11% of total time)
+ set +xe


The files passed all checks!

They are likely not order-dependent and can be run in random order and thus
are being removed from 'spec/support/rspec_order_todo.yml':

  * Removing 'ee/spec/services/vulnerability_scanners'

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Run `ee/spec/services/vulnerabilit*` specs in a random order

What does this MR do and why?

MR acceptance checklist

Merge request reports