Skip to content

Fix spring holding on to connections for multiple databases

Thong Kuah requested to merge fix_spring_multiple_databases into master

What does this MR do and why?

Spring only knows about the main database, so it correctly disconnects from that database. However, we need to do the same for the Ci database

Related issue: #356656 (closed)

To reproduce

This fixes bundle exec spring rspec spec/policies/project_policy_spec.rb when the test database needed to be reloaded.

  1. Setup multiple databases following #356353 (closed)

  2. Modify db/structure.sql to have something different like:

    +CREATE TABLE something(
    +    hello bigint
    +);
    +
  3. Run bundle exec spring rspec spec/policies/project_policy_spec.rb

On master:

$ be spring rspec spec/policies/project_policy_spec.rb
[TEST PROF INFO] Spring detected
Running via Spring preloader in process 81779
rails aborted!
ActiveRecord::StatementInvalid: PG::ObjectInUse: ERROR:  database "gitlabhq_test_ci" is being accessed by other users
DETAIL:  There is 1 other session using the database.
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/connection_proxy.rb:88:in `method_missing'

Caused by:
PG::ObjectInUse: ERROR:  database "gitlabhq_test_ci" is being accessed by other users
DETAIL:  There is 1 other session using the database.
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer'
/Users/tkuah/code/gdk-ee/gitlab/lib/gitlab/database/load_balancing/connection_proxy.rb:88:in `method_missing'
Tasks: TOP => db:test:load => db:test:purge
(See full trace by running task with --trace)

An error occurred while loading ./spec/policies/project_policy_spec.rb.
Failure/Error: ActiveRecord::Migration.maintain_test_schema!

ActiveRecord::PendingMigrationError:


  Migrations are pending. To resolve this issue, run:

          bin/rails db:migrate RAILS_ENV=test

  You have 1251 pending migrations:

  20210301200959_init_schema.rb
  20210302025305_add_excluded_urls_and_request_headers_to_dast_site_profiles.rb
  20210302074524_backfill_namespace_statistics_with_wiki_size.rb
  20210302103851_add_deployed_deployment_id_index_to_project_pages_metadata.rb
  20210302150310_schedule_migrate_pages_to_zip_storage.rb
  20210302155904_remove_index_for_security_orchestration_policy.rb

**snip

On this branch, we get:

$ be spring rspec spec/policies/project_policy_spec.rb
[TEST PROF INFO] Spring detected
Running via Spring preloader in process 82447
Run options: include {:focus=>true}

All examples were filtered out; ignoring {:focus=>true}

Test environment set up in 7.519957 seconds
.................................^C
RSpec is shutting down and will print the summary report... Interrupt again to force quit.
.

Finished in 16.1 seconds (files took 13.84 seconds to load)
34 examples, 0 failures

You can confirm we have something relation on test databases now:

gitlabhq_development=# \c gitlabhq_test_ci
You are now connected to database "gitlabhq_test_ci" as user "tkuah".
gitlabhq_test_ci=# \d something
             Table "public.something"
 Column |  Type  | Collation | Nullable | Default 
--------+--------+-----------+----------+---------
 hello  | bigint |           |          | 

gitlabhq_test_ci=# \c gitlabhq_test
You are now connected to database "gitlabhq_test" as user "tkuah".
gitlabhq_test=# \d something
             Table "public.something"
 Column |  Type  | Collation | Nullable | Default 
--------+--------+-----------+----------+---------
 hello  | bigint |           |          | 

Cleanup:

Run bundle exec rails db:test:prepare to cleanup

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Thong Kuah

Merge request reports