"PG::UndefinedObject: ERROR: index "index_taggings_on_taggable_id_and_taggable_type_and_context" does not exist - upgrade fail
Summary
Upgrading to 14.3.0 may fail with "PG::UndefinedObject: ERROR: index "index_taggings_on_taggable_id_and_taggable_type_and_context" does not exist
error.
Migration was introduced by this MR
Steps to reproduce
Example Project
NA
What is the current bug behavior?
Failed migration that may cause GitLab to be inoperable as reported by our Customer via internal ZD ticket
What is the expected correct behavior?
Relevant logs and/or screenshots
== 20210806131706 FinalizeTagginsBigintConversion: migrating ==================
-- transaction_open?()
-> 0.0000s
-- index_exists?("taggings", :id_convert_to_bigint, {:unique=>true, :name=>"index_taggings_on_id_convert_to_bigint", :algorithm=>:concurrently})
-> 0.0134s
-- transaction_open?()
-> 0.0000s
-- index_exists?("taggings", [:taggable_id_convert_to_bigint, :taggable_type], {:name=>"i_taggings_on_taggable_id_convert_to_bigint_and_taggable_type", :algorithm=>:concurrently})
-> 0.0108s
-- transaction_open?()
-> 0.0000s
-- index_exists?("taggings", [:taggable_id_convert_to_bigint, :taggable_type, :context], {:name=>"i_taggings_on_taggable_bigint_and_taggable_type_and_context", :algorithm=>:concurrently})
-> 0.0094s
-- transaction_open?()
-> 0.0000s
-- index_exists?("taggings", [:tag_id, :taggable_id_convert_to_bigint, :taggable_type, :context, :tagger_id, :tagger_type], {:unique=>true, :name=>"taggings_idx_tmp", :algorithm=>:concurrently})
-> 0.0119s
-- transaction_open?()
-> 0.0000s
-- index_exists?("taggings", :id_convert_to_bigint, {:where=>"taggable_type = 'Project'", :name=>"tmp_index_taggings_on_id_bigint_where_taggable_type_project", :algorithm=>:concurrently})
-> 0.0094s
-- execute("LOCK TABLE taggings IN ACCESS EXCLUSIVE MODE")
-> 0.0005s
-- quote_table_name("taggings")
-> 0.0001s
-- quote_column_name(:taggable_id)
-> 0.0000s
-- quote_column_name("taggable_id_tmp")
-> 0.0000s
-- execute("ALTER TABLE \"taggings\" RENAME COLUMN \"taggable_id\" TO \"taggable_id_tmp\"")
-> 0.0026s
-- quote_table_name("taggings")
-> 0.0000s
-- quote_column_name(:taggable_id_convert_to_bigint)
-> 0.0000s
-- quote_column_name(:taggable_id)
-> 0.0000s
-- execute("ALTER TABLE \"taggings\" RENAME COLUMN \"taggable_id_convert_to_bigint\" TO \"taggable_id\"")
-> 0.0005s
-- quote_table_name("taggings")
-> 0.0000s
-- quote_column_name("taggable_id_tmp")
-> 0.0000s
-- quote_column_name(:taggable_id_convert_to_bigint)
-> 0.0000s
-- execute("ALTER TABLE \"taggings\" RENAME COLUMN \"taggable_id_tmp\" TO \"taggable_id_convert_to_bigint\"")
-> 0.0004s
-- quote_table_name("taggings")
-> 0.0000s
-- quote_column_name(:id)
-> 0.0000s
-- quote_column_name("id_tmp")
-> 0.0000s
-- execute("ALTER TABLE \"taggings\" RENAME COLUMN \"id\" TO \"id_tmp\"")
-> 0.0006s
-- quote_table_name("taggings")
-> 0.0000s
-- quote_column_name(:id_convert_to_bigint)
-> 0.0000s
-- quote_column_name(:id)
-> 0.0000s
-- execute("ALTER TABLE \"taggings\" RENAME COLUMN \"id_convert_to_bigint\" TO \"id\"")
-> 0.0005s
-- quote_table_name("taggings")
-> 0.0000s
-- quote_column_name("id_tmp")
-> 0.0000s
-- quote_column_name(:id_convert_to_bigint)
-> 0.0000s
-- execute("ALTER TABLE \"taggings\" RENAME COLUMN \"id_tmp\" TO \"id_convert_to_bigint\"")
-> 0.0004s
-- quote_table_name("trigger_aebe8b822ad3")
-> 0.0001s
-- execute("ALTER FUNCTION \"trigger_aebe8b822ad3\" RESET ALL")
-> 0.0006s
-- execute("ALTER SEQUENCE taggings_id_seq OWNED BY taggings.id")
-> 0.0027s
-- change_column_default("taggings", :id, #<Proc:0x00007f9b8d18c088 /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:68 (lambda)>)
-> 0.0114s
-- change_column_default("taggings", :id_convert_to_bigint, 0)
-> 0.0055s
-- execute("ALTER TABLE taggings DROP CONSTRAINT taggings_pkey CASCADE")
-> 0.0015s
-- rename_index("taggings", "index_taggings_on_id_convert_to_bigint", "taggings_pkey")
-> 0.0011s
-- execute("ALTER TABLE taggings ADD CONSTRAINT taggings_pkey PRIMARY KEY USING INDEX taggings_pkey")
-> 0.0029s
-- execute("DROP INDEX index_taggings_on_taggable_id_and_taggable_type")
-> 0.0007s
-- rename_index("taggings", "i_taggings_on_taggable_id_convert_to_bigint_and_taggable_type", "index_taggings_on_taggable_id_and_taggable_type")
-> 0.0010s
-- execute("DROP INDEX index_taggings_on_taggable_id_and_taggable_type_and_context")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:
PG::UndefinedObject: ERROR: index "index_taggings_on_taggable_id_and_taggable_type_and_context" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:80:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:123:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:134:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:129:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:97:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:422:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:44:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
ActiveRecord::StatementInvalid: PG::UndefinedObject: ERROR: index "index_taggings_on_taggable_id_and_taggable_type_and_context" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:80:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:123:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:134:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:129:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:97:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:422:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:44:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
PG::UndefinedObject: ERROR: index "index_taggings_on_taggable_id_and_taggable_type_and_context" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:80:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:123:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:134:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:129:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:97:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:422:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:44:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210806131706_finalize_taggins_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)
Possible fixes
A similar problem was already resolved a couple of years ago by adding IF EXISTS
before DROP
statement. It looks now that we are trying to remove indexes without first checking if the table exists as seen here.
execute 'DROP INDEX index_taggings_on_taggable_id_and_taggable_type_and_context'
Changing it to
execute 'DROP INDEX IF EXISTS index_taggings_on_taggable_id_and_taggable_type_and_context'
would solve this problem for cases, where this table was already dropped.