Skip to content

[RUN ALL RSPEC] Move core EE load balancing code to CE

Stan Hu requested to merge sh-port-ee-load-balancing-ce into master

What does this MR do?

This is transparent to the user, we are moving the core code for DB Load balancing from EE to Core.

Not yet moved is code that relies on DB Load Balancing (see #327902 (comment 566635697)). This will be done in subsequent MR

Testing

bin/rspec spec/lib/gitlab/database/load_balancing/* spec/lib/gitlab/database/load_balancing_spec.rb

FOSS_ONLY=1 bin/rspec spec/lib/gitlab/database/load_balancing/* spec/lib/gitlab/database/load_balancing_spec.rb

Related issue: #327902 (closed)

Screenshots (strongly suggested)

EE

Manually tested by running locally with load_balancing configured in config/database.yml:

 11   load_balancing:                                                                                                                                     
 12     hosts:                                                                                                                                            
 13       - /Users/tkuah/code/ee-gdk/postgresql                                                                                                           
 14       - /Users/tkuah/code/ee-gdk/postgresql                                                                                                           
 15       - /Users/tkuah/code/ee-gdk/postgresql 

Screen_Shot_2021-05-25_at_12.47.24_PM

  (1.5ms)  SELECT pg_current_wal_insert_lsn()::text AS location
 /*application:web,correlation_id:01F6GFRPXGDH4EK5SMA1T9QG7N,line:/lib/gitlab/database.rb:315:in `get_write_location'*/
  ↳ lib/gitlab/database.rb:315:in `get_write_location'
Started GET "/" for 127.0.0.1 at 2021-05-25 12:46:37 +1200
   (7.2ms)  SELECT EXTRACT(EPOCH FROM (now() - pg_last_xact_replay_timestamp()))::float as lag /*application:web,correlation_id:01F6GFSQ9YFE2P1X586G1QB9VN,line:/lib/gitlab/database/load_balancing/host.rb:200:in `query_and_release'*/
  ↳ lib/gitlab/database/load_balancing/host.rb:200:in `query_and_release'
  ApplicationSetting Load (16.2ms)  SELECT "application_settings".* FROM "application_settings" ORDER BY "application_settings"."id" DESC LIMIT 1 /*application:web,correlation_id:01F6GFSQ9YFE2P1X586G1QB9VN,line:/lib/gitlab/database/load_balancing/connection_proxy.rb:94:in `block in read_using_load_balancer'*/
  ↳ lib/gitlab/database/load_balancing/connection_proxy.rb:94:in `block in read_using_load_balancer'
[1] pry(main)> ActiveRecord::Base.connection.class
=> Gitlab::Database::LoadBalancing::ConnectionProxy
[2] pry(main)> ActiveRecord::Base.connection
=> #<Gitlab::Database::LoadBalancing::ConnectionProxy:0x00007fd42be4ded0
 @load_balancer=
  #<Gitlab::Database::LoadBalancing::LoadBalancer:0x00007fd41cded440
   @connection_db_roles={},
   @connection_db_roles_count={},
   @host_list=
    #<Gitlab::Database::LoadBalancing::HostList:0x00007fd42be3e138
     @hosts=
      [#<Gitlab::Database::LoadBalancing::Host:0x00007fd42be45ff0

FOSS

Deleted the ee/ directory, and I don't see load balancing queries anymore :

Screen_Shot_2021-05-25_at_8.49.38_PM

=> #<ActiveRecord::ConnectionAdapters::PostgreSQLAdapter:0x00007fe8cff4d318
 @advisory_locks_enabled=true,
 @config=
  {:adapter=>"postgresql",
   :encoding=>"unicode",
   :database=>"gitlabhq_development",
   :host=>"/Users/tkuah/code/ee-gdk/postgresql",
   :port=>5432,
   :pool=>11,
   :prepared_statements=>false,
   :variables=>{"statement_timeout"=>"120s"},
   :load_balancing=>{"hosts"=>["/Users/tkuah/code/ee-gdk/postgresql", "/Users/tkuah/code/ee-gdk/postgresql", "/Users/tkuah/code/ee-gdk/postgresql"]}},

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by Thong Kuah

Merge request reports