Error migrating from 14.0.12 to 14.8.2
Sumary
Error migrating from 14.0.12 to 14.8.2: batched_background_migration_job_transition_logs" does not exist
There was an error running gitlab-ctl reconfigure: rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of "bash" "/tmp/chef-script20220228-16729-1d1rc1x" ---- STDOUT: rake aborted! StandardError: An error has occurred, all later migrations canceled: Expected batched background migration for the given configuration to be marked as 'finished', but it is 'finalizing': {:job_class_name=>"CopyColumnUsingBackgroundMigrationJob", :table_name=>"ci_builds", :column_name=>"id", :job_arguments=>[["id", "stage_id"], ["id_convert_to_bigint", "stage_id_convert_to_bigint"]]} Finalize it manualy by running sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_builds,id,'[["id"\, "stage_id"]\, ["id_convert_to_bigint"\, "stage_id_convert_to_bigint"]]'] For more information, check the documentation https://docs.gitlab.com/ee/user/admin_area/monitoring/background_migrations.html#database-migrations-failing-because-of-batched-background-migration-not-finished
When trying to run, what it specifies I get :
sudo gitlab-rake gitlab:background_migrations:finalize[CopyColumnUsingBackgroundMigrationJob,ci_builds,id,'[["id"\, "stage_id"]\, ["id_convert_to_bigint"\, "stage_id_convert_to_bigint"]]'] rake aborted! ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "batched_background_migration_job_transition_logs" does not exist LINE 8: WHERE a.attrelid = '"batched_background_migration_job_trans... ^ /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/schema_cache_with_renamed_table.rb:25:in `columns' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/schema_cache_with_renamed_table.rb:29:in `columns_hash' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_job.rb:59:in `block (2 levels) in ' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:285:in `block in transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:284:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:27:in `rescue in perform' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:26:in `perform' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:70:in `block in finalize' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:70:in `finalize' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:10:in `finalize' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/background_migrations.rake:14:in `block (3 levels) in ' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `' Caused by: PG::UndefinedTable: ERROR: relation "batched_background_migration_job_transition_logs" does not exist
If I run commands to check how many migrations are left:
gitlab-rails runner -e production 'puts Gitlab::Database::BackgroundMigrationJob.pending.count' 0 gitlab-rails runner -e production 'puts Gitlab::BackgroundMigration.remaining' 0
If I run query in DB:
sudo gitlab-psql psql (12.7, server 12.6) Type "help" for help. gitlabhq_production=# select job_class_name, table_name, column_name, job_arguments from batched_background_migrations where status <> 3; job_class_name | table_name | column_name | job_arguments ---------------------------------------+--------------------+-------------+------------------------------------ CopyColumnUsingBackgroundMigrationJob | ci_stages | id | [["id"], ["id_convert_to_bigint"]] CopyColumnUsingBackgroundMigrationJob | ci_builds_metadata | id | [["id"], ["id_convert_to_bigint"]] (2 rows)
Edited by Luka