Fix flaky vault_spec
What does this merge request do and why?
It seems that using real GDK root and accessing GDK.config doesn't play well and cause flaky specs.
The reason is that GDK.config is cached which includes the unstubbed GDK root (config.gdk_root).
Ideally, we would pass GDK root explicitly to config.
However, in this case :gdk_root
is not even needed as all file access
is stubbed as well.
Refs #2125
How to set up and validate locally
# On main
bundle exec rspec --format documentation --order defined spec/lib/gdk/announcements_spec.rb spec/gdk/services/vault_spec.rb
GDK::Announcements
#render_all
renders an announcement to the terminal
with multiple announcements
renders all announcements to the terminal
GDK::Services::Vault
#name
returns vault
#command
returns command based on config (FAILED - 1)
#enabled?
returns true if set `enabled: true` in the config file
returns false if set `enabled: false` in the config file
Failures:
1) GDK::Services::Vault#command returns command based on config
Failure/Error: expect(subject.command).to match(/vault server --dev --dev-listen-address=127.0.0.1:8200/)
expected "/usr/local/bin/vault server --dev --dev-listen-address=10.23.0.5:8200" to match /vault server --dev --dev-listen-address=127.0.0.1:8200/
Diff:
@@ -1 +1 @@
-/vault server --dev --dev-listen-address=127.0.0.1:8200/
+"/usr/local/bin/vault server --dev --dev-listen-address=10.23.0.5:8200"
# ./spec/gdk/services/vault_spec.rb:12:in `block (3 levels) in <top (required)>'
# spec/spec_support/system_exit_detected.rb:7:in `block (2 levels) in <top (required)>'
Finished in 0.0584 seconds (files took 0.21079 seconds to load)
6 examples, 1 failure
Failed examples:
rspec ./spec/gdk/services/vault_spec.rb:11 # GDK::Services::Vault#command returns command based on config
Coverage report generated for RSpec to /home/peter/devel/gitlab/gdk/coverage. 508 / 1406 LOC (36.13%) covered.
Coverage report generated for RSpec to /home/peter/devel/gitlab/gdk/coverage/coverage.xml. 508 / 1406 LOC (36.13%) covered
Stopped processing SimpleCov as a previous error not related to SimpleCov has been detected
Found via RSpec bisect:
bundle exec rspec --bisect --seed 61856
Bisect started using options: "--seed 61856"
Running suite to find failures... (6.74 seconds)
Starting bisect with 5 failing examples and 1284 non-failing examples.
Checking that failure(s) are order-dependent... failure appears to be order-dependent
Round 1: bisecting over non-failing examples 1-1284 .. ignoring examples 643-1284 (0.79865 seconds)
Round 2: bisecting over non-failing examples 1-642 .. ignoring examples 322-642 (0.72283 seconds)
Round 3: bisecting over non-failing examples 1-321 .. ignoring examples 162-321 (0.59895 seconds)
Round 4: bisecting over non-failing examples 1-161 .. ignoring examples 82-161 (0.42429 seconds)
Round 5: bisecting over non-failing examples 1-81 .. ignoring examples 42-81 (0.30834 seconds)
Round 6: bisecting over non-failing examples 1-41 .. ignoring examples 22-41 (0.24826 seconds)
Round 7: bisecting over non-failing examples 1-21 .. ignoring examples 12-21 (0.22735 seconds)
Round 8: bisecting over non-failing examples 1-11 .. ignoring examples 7-11 (0.19596 seconds)
Round 9: bisecting over non-failing examples 1-6 .. ignoring examples 4-6 (0.19477 seconds)
Round 10: bisecting over non-failing examples 1-3 .. ignoring example 3 (0.17767 seconds)
Round 11: bisecting over non-failing examples 1-2 . ignoring example 1 (0.09199 seconds)
Bisect complete! Reduced necessary non-failing examples from 1284 to 1 in 4.07 seconds.
The minimal reproduction command is:
rspec ./spec/gdk/services/vault_spec.rb[1:2:1] ./spec/lib/gdk/announcements_spec.rb[1:1:2:1] ./spec/lib/gdk/services/gitlab_workhorse_spec.rb[1:2:1] ./spec/lib/gdk/services/minio_spec.rb[1:2:1] ./spec/lib/gdk/services/redis_cluster_spec.rb[1:2:1] ./spec/lib/gdk/services/vault_spec.rb[1:2:1] --seed 61856
Coverage report generated for RSpec to /home/peter/devel/gitlab/gdk/coverage. 23 / 33 LOC (69.7%) covered.
Coverage report generated for RSpec to /home/peter/devel/gitlab/gdk/coverage/coverage.xml. 23 / 33 LOC (69.70%) covered
All specs pass on this branch:
bundle exec rspec ./spec/gdk/services/vault_spec.rb[1:2:1] ./spec/lib/gdk/announcements_spec.rb[1:1:2:1] ./spec/lib/gdk/services/gitlab_workhorse_spec.rb[1:2:1] ./spec/lib/gdk/services/minio_spec.rb[1:2:1] ./spec/lib/gdk/services/redis_cluster_spec.rb[1:2:1] ./spec/lib/gdk/services/vault_spec.rb[1:2:1] --seed 61856
Run options: include {:ids=>{"./spec/gdk/services/vault_spec.rb"=>["1:2:1"], "./spec/lib/gdk/announcements_spec.rb"=>["1:1:2:1"], "./spec/lib/gdk/services/gitlab_workhorse_spec.rb"=>["1:2:1"], "./spec/lib/gdk/services/minio_spec.rb"=>["1:2:1"], "./spec/lib/gdk/services/redis_cluster_spec.rb"=>["1:2:1"], "./spec/lib/gdk/services/vault_spec.rb"=>["1:2:1"]}}
Randomized with seed 61856
......
Finished in 0.05516 seconds (files took 0.21513 seconds to load)
6 examples, 0 failures
Randomized with seed 61856
Coverage report generated for RSpec to /home/peter/devel/gitlab/gdk/coverage. 702 / 1553 LOC (45.2%) covered.
Coverage report generated for RSpec to /home/peter/devel/gitlab/gdk/coverage/coverage.xml. 702 / 1553 LOC (45.20%) covered
Impacted categories
The following categories relate to this merge request:
-
gdk-reliability - e.g. When a GDK action fails to complete. -
gdk-usability - e.g. Improvements or suggestions around how the GDK functions. -
gdk-performance - e.g. When a GDK action is slow or times out.
Merge request checklist
-
This change is backward compatible. If not, please include steps to communicate to our users. -
Tests added for new functionality. If not, please raise an issue to follow-up. -
Documentation added/updated, if needed. -
Announcement added, if change is notable. -
gdk doctor
test added, if needed. -
Add the ~highlight
label if this MR should be included in theCHANGELOG.md
.
Edited by Peter Leitzen