Skip to content

Fix rename_column_concurrently to work with decomposed DB

Omar Qunsul requested to merge 367254-fix-rename-column-concurrently into master

What does this MR do and why?

#367254 (closed)

How to set up and validate locally

  1. Make sure you are running on Decomposed Database Setup #356353 (closed)

  2. Fill the users table on the main database with some data.

./bin/rails c
[1] pry(main)> FactoryBot.create_list(:user, 50)

then exit the Rails console.

  1. Copy the data of the Users table from the main database to the ci database. Make sure you adjust the hosts parameters to the hosts of your database. See db/config.yml for the actual hosts.
pg_dump -h /Users/omar/gdk/postgresql -t users -d gitlabhq_development > ~/main.sql
psql -h /Users/omar/gdk/postgresql -d gitlabhq_development_ci < ~/main.sql
  1. Now look the main tables on ci database, and ci tables on the main database, using this rake task
rake gitlab:db:lock_writes
  1. Create a Test Database Migration to rename a column
rails g migration test_migration

and write this code for the new DB migration

class TestMigration < Gitlab::Database::Migration[2.0]
  disable_ddl_transaction!

  def up
    rename_column_concurrently :users, :role, :new_role
  end

  def down
    undo_rename_column_concurrently :users, :new_role, :role
  end
end

Then run rake db:migrate

If you were running this on the master branch, you would get this error

Caused by:
ActiveRecord::StatementInvalid: PG::SREModifyingSqlDataNotPermitted: ERROR:  Table: "users" is write protected within this Gitlab database.
HINT:  Make sure you are using the right database connection
CONTEXT:  PL/pgSQL function gitlab_schema_prevent_write() line 4 at RAISE
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers.rb:562:in `block in update_column_in_batches'
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers.rb:539:in `loop'
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers.rb:539:in `update_column_in_batches'
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers.rb:1670:in `block (2 levels) in create_column_from'
/Users/omar/gdk/gitlab/lib/gitlab/database/query_analyzers/base.rb:25:in `with_suppressed'
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers.rb:1669:in `block in create_column_from'
/Users/omar/gdk/gitlab/lib/gitlab/database/query_analyzers/base.rb:25:in `with_suppressed'
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers.rb:1668:in `create_column_from'
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers/v2.rb:219:in `setup_renamed_column'
/Users/omar/gdk/gitlab/lib/gitlab/database/migration_helpers/v2.rb:139:in `rename_column_concurrently'

Which is what we are solving in this MR.

But if you ran the migration on this branch, you should see

ci: == 20220726204835 TestMigration: migrated (0.0104s) ===========================

and no errors 🎉

  1. To validate this
gdk psql -d gitlabhq_development_ci
gitlabhq_development_ci=# select role, new_role from users;
 role | new_role 
------+----------
    0 |        0
    0 |        0
    0 |        0
    0 |        0
    0 |        0
    0 |        0

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #367254 (closed)

Edited by Omar Qunsul

Merge request reports