Fix flakiness in current_settings_reader specs
What does this MR do and why?
Fix flakiness in current_settings_reader specs related to intermittent cache-related local failures in Gitlab::CurrentSettings
Occasionally when running local tests after a GDK update, you get errors like the following. They are not reproducible, the tests pass when run a second time.
This MR attempts to avoid those.
EXAMPLE OF ERRORS IN TEST FAILURE OUTPUT
Failures:
1) RemoteDevelopment::Settings::CurrentSettingsReader when no relevant settings are requested returns an OK result containing the original context
Failure/Error: connection.public_send(...)
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column application_settings.default_max_hours_before_termination does not exist
LINE 1: ...porters", "application_settings"."code_creation", "applicati...
^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `block in read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:106:in `read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:48:in `select_all'
# ./gems/activerecord-gitlab/lib/active_record/gitlab_patches/rescue_from.rb:31:in `exec_queries'
# ./app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
# ./app/models/concerns/cacheable_attributes.rb:55:in `current'
# ./app/validators/addressable_url_validator.rb:126:in `allow_setting_local_requests?'
# ./app/validators/addressable_url_validator.rb:111:in `block in blocker_args'
# ./app/validators/addressable_url_validator.rb:110:in `blocker_args'
# ./app/validators/addressable_url_validator.rb:89:in `validate_each'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction'
# ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:31:in `block (2 levels) in <main>'
# ./spec/spec_helper.rb:477:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'
# ./spec/spec_helper.rb:476:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:471:in `block (3 levels) in <top (required)>'
# ./spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
# ./spec/spec_helper.rb:462:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:458:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/application_context.rb:100:in `with_raw_context'
# ./spec/spec_helper.rb:458:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:429:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/ci/config/feature_flags.rb:38:in `ensure_correct_usage'
# ./spec/spec_helper.rb:428:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:281:in `block (2 levels) in <top (required)>'
# ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
# ------------------
# --- Caused by: ---
# PG::UndefinedColumn:
# ERROR: column application_settings.default_max_hours_before_termination does not exist
# LINE 1: ...porters", "application_settings"."code_creation", "applicati...
# ^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
2) RemoteDevelopment::Settings::CurrentSettingsReader when a relevant setting is not a valid CurrentSettings entry raises a runtime error
Failure/Error: connection.public_send(...)
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column application_settings.default_max_hours_before_termination does not exist
LINE 1: ...porters", "application_settings"."code_creation", "applicati...
^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `block in read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:106:in `read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:48:in `select_all'
# ./gems/activerecord-gitlab/lib/active_record/gitlab_patches/rescue_from.rb:31:in `exec_queries'
# ./app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
# ./app/models/concerns/cacheable_attributes.rb:55:in `current'
# ./app/validators/addressable_url_validator.rb:126:in `allow_setting_local_requests?'
# ./app/validators/addressable_url_validator.rb:111:in `block in blocker_args'
# ./app/validators/addressable_url_validator.rb:110:in `blocker_args'
# ./app/validators/addressable_url_validator.rb:89:in `validate_each'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction'
# ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:31:in `block (2 levels) in <main>'
# ./spec/spec_helper.rb:477:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'
# ./spec/spec_helper.rb:476:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:471:in `block (3 levels) in <top (required)>'
# ./spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
# ./spec/spec_helper.rb:462:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:458:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/application_context.rb:100:in `with_raw_context'
# ./spec/spec_helper.rb:458:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:429:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/ci/config/feature_flags.rb:38:in `ensure_correct_usage'
# ./spec/spec_helper.rb:428:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:281:in `block (2 levels) in <top (required)>'
# ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
# ------------------
# --- Caused by: ---
# PG::UndefinedColumn:
# ERROR: column application_settings.default_max_hours_before_termination does not exist
# LINE 1: ...porters", "application_settings"."code_creation", "applicati...
# ^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
3) RemoteDevelopment::Settings::CurrentSettingsReader when the relevant settings are valid CurrentSettings entries when there are no errors returns ::Gitlab::CurrentSettings overridden settings as well as other non-relevant settings
Failure/Error: connection.public_send(...)
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column application_settings.default_max_hours_before_termination does not exist
LINE 1: ...porters", "application_settings"."code_creation", "applicati...
^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `block in read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:106:in `read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:48:in `select_all'
# ./gems/activerecord-gitlab/lib/active_record/gitlab_patches/rescue_from.rb:31:in `exec_queries'
# ./app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
# ./app/models/concerns/cacheable_attributes.rb:55:in `current'
# ./app/validators/addressable_url_validator.rb:126:in `allow_setting_local_requests?'
# ./app/validators/addressable_url_validator.rb:111:in `block in blocker_args'
# ./app/validators/addressable_url_validator.rb:110:in `blocker_args'
# ./app/validators/addressable_url_validator.rb:89:in `validate_each'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction'
# ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:31:in `block (2 levels) in <main>'
# ./spec/spec_helper.rb:477:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'
# ./spec/spec_helper.rb:476:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:471:in `block (3 levels) in <top (required)>'
# ./spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
# ./spec/spec_helper.rb:462:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:458:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/application_context.rb:100:in `with_raw_context'
# ./spec/spec_helper.rb:458:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:429:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/ci/config/feature_flags.rb:38:in `ensure_correct_usage'
# ./spec/spec_helper.rb:428:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:281:in `block (2 levels) in <top (required)>'
# ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
# ------------------
# --- Caused by: ---
# PG::UndefinedColumn:
# ERROR: column application_settings.default_max_hours_before_termination does not exist
# LINE 1: ...porters", "application_settings"."code_creation", "applicati...
# ^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
4) RemoteDevelopment::Settings::CurrentSettingsReader when the relevant settings are valid CurrentSettings entries when the type from GitLab::CurrentSettings does not match the declared remote development setting type returns an err Result containing a Gitlab::CurrentSettings read failed message with details
Failure/Error: connection.public_send(...)
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column application_settings.default_max_hours_before_termination does not exist
LINE 1: ...porters", "application_settings"."code_creation", "applicati...
^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `block in read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:106:in `read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:48:in `select_all'
# ./gems/activerecord-gitlab/lib/active_record/gitlab_patches/rescue_from.rb:31:in `exec_queries'
# ./app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
# ./app/models/concerns/cacheable_attributes.rb:55:in `current'
# ./app/validators/addressable_url_validator.rb:126:in `allow_setting_local_requests?'
# ./app/validators/addressable_url_validator.rb:111:in `block in blocker_args'
# ./app/validators/addressable_url_validator.rb:110:in `blocker_args'
# ./app/validators/addressable_url_validator.rb:89:in `validate_each'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction'
# ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:31:in `block (2 levels) in <main>'
# ./spec/spec_helper.rb:477:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'
# ./spec/spec_helper.rb:476:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:471:in `block (3 levels) in <top (required)>'
# ./spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
# ./spec/spec_helper.rb:462:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:458:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/application_context.rb:100:in `with_raw_context'
# ./spec/spec_helper.rb:458:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:429:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/ci/config/feature_flags.rb:38:in `ensure_correct_usage'
# ./spec/spec_helper.rb:428:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:281:in `block (2 levels) in <top (required)>'
# ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
# ------------------
# --- Caused by: ---
# PG::UndefinedColumn:
# ERROR: column application_settings.default_max_hours_before_termination does not exist
# LINE 1: ...porters", "application_settings"."code_creation", "applicati...
# ^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
5) RemoteDevelopment::Settings when there is and ENV var override and also a ::Gitlab::CurrentSettings override uses the ENV var value and not the CurrentSettings value
Failure/Error: connection.public_send(...)
ActiveRecord::StatementInvalid:
PG::UndefinedColumn: ERROR: column application_settings.default_max_hours_before_termination does not exist
LINE 1: ...porters", "application_settings"."code_creation", "applicati...
^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `block in read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:106:in `read_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:48:in `select_all'
# ./gems/activerecord-gitlab/lib/active_record/gitlab_patches/rescue_from.rb:31:in `exec_queries'
# ./app/models/concerns/cacheable_attributes.rb:19:in `current_without_cache'
# ./app/models/concerns/cacheable_attributes.rb:55:in `current'
# ./app/validators/addressable_url_validator.rb:126:in `allow_setting_local_requests?'
# ./app/validators/addressable_url_validator.rb:111:in `block in blocker_args'
# ./app/validators/addressable_url_validator.rb:110:in `blocker_args'
# ./app/validators/addressable_url_validator.rb:89:in `validate_each'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `public_send'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:127:in `block in write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:141:in `block in read_write'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:228:in `retry_with_backoff'
# ./lib/gitlab/database/load_balancing/load_balancer.rb:130:in `read_write'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:78:in `transaction'
# ./ee/spec/lib/remote_development/settings/settings_integration_spec.rb:42:in `block (3 levels) in <main>'
# ./spec/spec_helper.rb:477:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/sidekiq_sharding/validator.rb:42:in `enabled'
# ./spec/spec_helper.rb:476:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:471:in `block (3 levels) in <top (required)>'
# ./spec/support/sidekiq_middleware.rb:9:in `with_sidekiq_server_middleware'
# ./spec/spec_helper.rb:462:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:458:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/application_context.rb:100:in `with_raw_context'
# ./spec/spec_helper.rb:458:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:429:in `block (3 levels) in <top (required)>'
# ./lib/gitlab/ci/config/feature_flags.rb:38:in `ensure_correct_usage'
# ./spec/spec_helper.rb:428:in `block (2 levels) in <top (required)>'
# ./spec/spec_helper.rb:281:in `block (2 levels) in <top (required)>'
# ./spec/support/system_exit_detected.rb:7:in `block (2 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (3 levels) in <main>'
# ./spec/support/database/prevent_cross_joins.rb:60:in `with_cross_joins_prevented'
# ./spec/support/database/prevent_cross_joins.rb:106:in `block (2 levels) in <main>'
# ------------------
# --- Caused by: ---
# PG::UndefinedColumn:
# ERROR: column application_settings.default_max_hours_before_termination does not exist
# LINE 1: ...porters", "application_settings"."code_creation", "applicati...
# ^
# ./lib/gitlab/database/load_balancing/connection_proxy.rb:107:in `public_send'
Finished in 7 minutes 5 seconds (files took 1 minute 3.71 seconds to load)
1053 examples, 5 failures
Failed examples:
rspec ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:82 # RemoteDevelopment::Settings::CurrentSettingsReader when no relevant settings are requested returns an OK result containing the original context
rspec ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:88 # RemoteDevelopment::Settings::CurrentSettingsReader when a relevant setting is not a valid CurrentSettings entry raises a runtime error
rspec ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:40 # RemoteDevelopment::Settings::CurrentSettingsReader when the relevant settings are valid CurrentSettings entries when there are no errors returns ::Gitlab::CurrentSettings overridden settings as well as other non-relevant settings
rspec ./ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb:59 # RemoteDevelopment::Settings::CurrentSettingsReader when the relevant settings are valid CurrentSettings entries when the type from GitLab::CurrentSettings does not match the declared remote development setting type returns an err Result containing a Gitlab::CurrentSettings read failed message with details
rspec ./ee/spec/lib/remote_development/settings/settings_integration_spec.rb:46 # RemoteDevelopment::Settings when there is and ENV var override and also a ::Gitlab::CurrentSettings override uses the ENV var value and not the CurrentSettings value
References
ee/spec/lib/remote_development/settings/current_settings_reader_spec.rb
Screenshots or screen recordings
See test failure output above.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Edited by Chad Woolley