Upgrade from 14.2.2 to 14.4.2 ce fails because of missing table
Summary
I'm failing to upgrade Gitlab-CE from version 14.2.2 to 14.4.2.
I throws the error "PG::UndefinedTable: ERROR: relation "events_id_seq" does not exist".
I already tried to restore a backup from the old version and tried again - without success.
Steps to reproduce
- Start the upgrade
- Watch it unpacking
- Watch it failing while trying to reconfigure.
What is the current bug behavior?
Upgrade fails with "PG::UndefinedTable: ERROR: relation "events_id_seq" does not exist" message.
What is the expected correct behavior?
Upgrade should be successful.
Relevant logs and/or screenshots
There was an error running gitlab-ctl reconfigure:
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/tmp/chef-script20211112-6449-1n5ots0" ----
STDOUT: rake aborted!
StandardError: An error has occurred, all later migrations canceled:
PG::UndefinedTable: ERROR: relation "events_id_seq" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:67:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:123:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:134:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:170:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:108:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:129:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:97:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:422:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:44:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR: relation "events_id_seq" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:67:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:123:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:134:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:170:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:108:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:129:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:97:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:422:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:44:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
PG::UndefinedTable: ERROR: relation "events_id_seq" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:67:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:123:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:134:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:119:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:170:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:108:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:118:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:70:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:129:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:97:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:422:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:44:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210622045705_finalize_events_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:61:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20210622045705 FinalizeEventsBigintConversion: migrating ===================
-- transaction_open?()
-> 0.0000s
-- index_exists?("events", :id_convert_to_bigint, {:unique=>true, :name=>"index_events_on_id_convert_to_bigint", :algorithm=>:concurrently})
-> 0.0143s
-- transaction_open?()
-> 0.0000s
-- index_exists?("events", [:project_id, :id_convert_to_bigint], {:name=>"index_events_on_project_id_and_id_convert_to_bigint", :algorithm=>:concurrently})
-> 0.0120s
-- transaction_open?()
-> 0.0000s
-- index_exists?("events", [:project_id, :id_convert_to_bigint], {:order=>{:id_convert_to_bigint=>:desc}, :where=>"action = 7", :name=>"index_events_on_project_id_and_id_bigint_desc_on_merged_action", :algorithm=>:concurrently})
-> 0.0091s
-- transaction_open?()
-> 0.0000s
-- foreign_keys(:push_event_payloads)
-> 0.0092s
-- execute("SET statement_timeout TO 0")
-> 0.0002s
-- execute("ALTER TABLE push_event_payloads VALIDATE CONSTRAINT fk_36c74129da_tmp;")
-> 0.0006s
-- execute("RESET statement_timeout")
-> 0.0003s
-- execute("LOCK TABLE events, push_event_payloads IN ACCESS EXCLUSIVE MODE")
-> 0.0006s
-- quote_table_name("events")
-> 0.0001s
-- quote_column_name(:id)
-> 0.0000s
-- quote_column_name("id_tmp")
-> 0.0000s
-- execute("ALTER TABLE \"events\" RENAME COLUMN \"id\" TO \"id_tmp\"")
-> 0.0006s
-- quote_table_name("events")
-> 0.0000s
-- quote_column_name(:id_convert_to_bigint)
-> 0.0000s
-- quote_column_name(:id)
-> 0.0000s
-- execute("ALTER TABLE \"events\" RENAME COLUMN \"id_convert_to_bigint\" TO \"id\"")
-> 0.0005s
-- quote_table_name("events")
-> 0.0000s
-- quote_column_name("id_tmp")
-> 0.0000s
-- quote_column_name(:id_convert_to_bigint)
-> 0.0000s
-- execute("ALTER TABLE \"events\" RENAME COLUMN \"id_tmp\" TO \"id_convert_to_bigint\"")
-> 0.0004s
-- quote_table_name("trigger_69523443cc10")
-> 0.0000s
-- execute("ALTER FUNCTION \"trigger_69523443cc10\" RESET ALL")
-> 0.0007s
-- execute("ALTER SEQUENCE events_id_seq OWNED BY events.id")
STDERR:
---- End output of "bash" "/tmp/chef-script20211112-6449-1n5ots0" ----
Ran "bash" "/tmp/chef-script20211112-6449-1n5ots0" returned 1Results of GitLab environment info
Debian 10
Possible fixes
Restore the missing table(s).