DB migration fails after Omnibus upgrade to 14.2.2 with PG::UndefinedObject Error
Summary
During upgrade to 14.2.2, the gitlab-rake db:migrate
task fails with the following error:
-- execute("DROP INDEX index_ci_sources_pipelines_on_source_job_id")
rake aborted!
StandardError: An error has occurred, all later migrations canceled:
PG::UndefinedObject: ERROR: index "index_ci_sources_pipelines_on_source_job_id" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:57:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:121:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:130:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:214:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:213:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:125:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:95:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:418:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:36:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
ActiveRecord::StatementInvalid: PG::UndefinedObject: ERROR: index "index_ci_sources_pipelines_on_source_job_id" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:57:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:121:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:130:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:214:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:213:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:125:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:95:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:418:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:36:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
PG::UndefinedObject: ERROR: index "index_ci_sources_pipelines_on_source_job_id" does not exist
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:57:in `block in swap'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:121:in `run_block'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:130:in `block in run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:214:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:213:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:125:in `run_block_with_lock_timeout'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:95:in `run'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:418:in `with_lock_retries'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:36:in `swap'
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:18:in `up'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
This was also reported (with a solution) at https://forum.gitlab.com/t/database-migration-failed-during-update-to-14-2-2/57889
Steps to reproduce
Zero downtime upgrade to 14.2.2 (in my case from 14.1.1) on CentOS.
touch /etc/gitlab/skip-auto-reconfigure
yum upgrade gitlab-ee
SKIP_POST_DEPLOYMENT_MIGRATIONS=true gitlab-ctl reconfigure
gitlab-rake db:migrate
Example Project
What is the current bug behavior?
DB Migration fails with an error
What is the expected correct behavior?
DB Migration handles the error and continues
Relevant logs and/or screenshots
gitlab-rake db:migrate --trace
Expand for output
-- execute("DROP INDEX index_ci_sources_pipelines_on_source_job_id") rake aborted! StandardError: An error has occurred, all later migrations canceled: PG::UndefinedObject: ERROR: index "index_ci_sources_pipelines_on_source_job_id" does not exist /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:47:in `exec' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:47:in `block (2 levels) in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:46:in `block in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:45:in `execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/marginalia-1.10.0/lib/marginalia.rb:71:in `execute_with_marginalia' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:929:in `block in method_missing' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:897:in `block in say_with_time' /opt/gitlab/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:897:in `say_with_time' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:918:in `method_missing' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:57:in `block in swap' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:121:in `run_block' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:130:in `block in run_block_with_lock_timeout' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `block in transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in `block in within_new_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in `within_new_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:214:in `block in transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/notifications.rb:205:in `instrument' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:213:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:125:in `run_block_with_lock_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:95:in `run' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:418:in `with_lock_retries' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:36:in `swap' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:18:in `up' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:870:in `public_send' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:870:in `exec_migration' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:851:in `block (2 levels) in migrate' /opt/gitlab/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:850:in `block in migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:849:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1037:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1329:in `block in execute_migration_in_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1382:in `ddl_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1328:in `execute_migration_in_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1302:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1302:in `migrate_without_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1251:in `block in migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1401:in `block in with_advisory_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1416:in `block in with_advisory_lock_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1416:in `with_advisory_lock_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1397:in `with_advisory_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1251:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1086:in `up' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1061:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/tasks/database_tasks.rb:237:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:92:in `block (3 levels) in ' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:90:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:90:in `block (2 levels) in ' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/exe/rake:27:in `' /opt/gitlab/embedded/bin/rake:23:in `load' /opt/gitlab/embedded/bin/rake:23:in `' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `load' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `kernel_load' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:28:in `run' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:476:in `exec' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor.rb:399:in `dispatch' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:30:in `dispatch' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/base.rb:476:in `start' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:24:in `start' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/exe/bundle:46:in `block in ' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/friendly_errors.rb:123:in `with_friendly_errors' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/exe/bundle:34:in `' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `' Caused by: ActiveRecord::StatementInvalid: PG::UndefinedObject: ERROR: index "index_ci_sources_pipelines_on_source_job_id" does not exist /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:47:in `exec' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:47:in `block (2 levels) in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:46:in `block in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:45:in `execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/marginalia-1.10.0/lib/marginalia.rb:71:in `execute_with_marginalia' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:929:in `block in method_missing' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:897:in `block in say_with_time' /opt/gitlab/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:897:in `say_with_time' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:918:in `method_missing' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:57:in `block in swap' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:121:in `run_block' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:130:in `block in run_block_with_lock_timeout' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `block in transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in `block in within_new_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in `within_new_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:214:in `block in transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/notifications.rb:205:in `instrument' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:213:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:125:in `run_block_with_lock_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:95:in `run' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:418:in `with_lock_retries' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:36:in `swap' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:18:in `up' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:870:in `public_send' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:870:in `exec_migration' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:851:in `block (2 levels) in migrate' /opt/gitlab/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:850:in `block in migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:849:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1037:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1329:in `block in execute_migration_in_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1382:in `ddl_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1328:in `execute_migration_in_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1302:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1302:in `migrate_without_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1251:in `block in migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1401:in `block in with_advisory_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1416:in `block in with_advisory_lock_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1416:in `with_advisory_lock_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1397:in `with_advisory_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1251:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1086:in `up' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1061:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/tasks/database_tasks.rb:237:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:92:in `block (3 levels) in ' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:90:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:90:in `block (2 levels) in ' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/exe/rake:27:in `' /opt/gitlab/embedded/bin/rake:23:in `load' /opt/gitlab/embedded/bin/rake:23:in `' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `load' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `kernel_load' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:28:in `run' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:476:in `exec' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor.rb:399:in `dispatch' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:30:in `dispatch' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/base.rb:476:in `start' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:24:in `start' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/exe/bundle:46:in `block in ' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/friendly_errors.rb:123:in `with_friendly_errors' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/exe/bundle:34:in `' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `' Caused by: PG::UndefinedObject: ERROR: index "index_ci_sources_pipelines_on_source_job_id" does not exist /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:47:in `exec' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:47:in `block (2 levels) in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/share_lock.rb:187:in `yield_shares' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:46:in `block in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:696:in `block (2 levels) in log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:695:in `block in log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/notifications/instrumenter.rb:24:in `instrument' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract_adapter.rb:687:in `log' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/postgresql/database_statements.rb:45:in `execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/marginalia-1.10.0/lib/marginalia.rb:71:in `execute_with_marginalia' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:929:in `block in method_missing' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:897:in `block in say_with_time' /opt/gitlab/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:897:in `say_with_time' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:918:in `method_missing' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:57:in `block in swap' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:121:in `run_block' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:130:in `block in run_block_with_lock_timeout' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `block in transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in `block in within_new_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in `within_new_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:214:in `block in transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/notifications.rb:205:in `instrument' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:213:in `transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:125:in `run_block_with_lock_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/with_lock_retries.rb:95:in `run' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:418:in `with_lock_retries' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:36:in `swap' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb:18:in `up' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:870:in `public_send' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:870:in `exec_migration' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:851:in `block (2 levels) in migrate' /opt/gitlab/embedded/lib/ruby/2.7.0/benchmark.rb:293:in `measure' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:850:in `block in migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:849:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1037:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1329:in `block in execute_migration_in_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1382:in `ddl_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1328:in `execute_migration_in_transaction' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1302:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1302:in `migrate_without_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1251:in `block in migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1401:in `block in with_advisory_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1416:in `block in with_advisory_lock_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/connection_pool.rb:462:in `with_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1416:in `with_advisory_lock_connection' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1397:in `with_advisory_lock' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1251:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1086:in `up' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/migration.rb:1061:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/tasks/database_tasks.rb:237:in `migrate' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:92:in `block (3 levels) in ' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:90:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/railties/databases.rake:90:in `block (2 levels) in ' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `block in execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:281:in `execute' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:219:in `block in invoke_with_call_chain' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `synchronize' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:199:in `invoke_with_call_chain' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/task.rb:188:in `invoke' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:160:in `invoke_task' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block (2 levels) in top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `each' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:116:in `block in top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:125:in `run_with_threads' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:110:in `top_level' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:83:in `block in run' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:186:in `standard_exception_handling' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/lib/rake/application.rb:80:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/rake-13.0.6/exe/rake:27:in `' /opt/gitlab/embedded/bin/rake:23:in `load' /opt/gitlab/embedded/bin/rake:23:in `' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `load' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:63:in `kernel_load' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli/exec.rb:28:in `run' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:476:in `exec' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/command.rb:27:in `run' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor.rb:399:in `dispatch' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:30:in `dispatch' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/vendor/thor/lib/thor/base.rb:476:in `start' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/cli.rb:24:in `start' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/exe/bundle:46:in `block in ' /opt/gitlab/embedded/lib/ruby/site_ruby/2.7.0/bundler/friendly_errors.rb:123:in `with_friendly_errors' /opt/gitlab/embedded/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/exe/bundle:34:in `' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `' Tasks: TOP => db:migrate
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
# gitlab-rake gitlab:env:info System information System: Proxy: no Current User: git Using RVM: no Ruby Version: 2.7.2p137 Gem Version: 3.1.4 Bundler Version:2.1.4 Rake Version: 13.0.6 Redis Version: 6.0.14 Git Version: 2.32.0 Sidekiq Version:5.2.9 Go Version: unknown GitLab information Version: 14.2.2-ee Revision: 406469ced21 Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: PostgreSQL DB Version: 12.6 URL: https://innersource.exfo.com HTTP Clone URL: https://innersource.exfo.com/some-group/some-project.git SSH Clone URL: git@innersource.exfo.com:some-group/some-project.git Elasticsearch: no Geo: no Using LDAP: yes Using Omniauth: yes Omniauth Providers: GitLab Shell Version: 13.19.1 Repository storage paths: - default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.19.1 ? ... OK (13.19.1) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Internal API available: OK Redis available via internal API: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes Number of Sidekiq processes (cluster/worker) ... 1/1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain LDAP authentication... Success LDAP users with access to your GitLab server (only showing the first 100 results) User output sanitized. Found 100 users of 100 limit.
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 2/1 ... yes 40/2 ... yes 15/3 ... yes 15/4 ... yes 15/5 ... yes 2/8 ... yes 2/9 ... yes 2/10 ... yes 15/11 ... yes 15/12 ... yes 15/13 ... yes 15/15 ... yes 15/16 ... yes 15/17 ... yes 2/18 ... yes 15/19 ... yes 15/22 ... yes 15/23 ... yes 15/24 ... yes 15/25 ... yes 33/26 ... yes 28/27 ... yes 38/28 ... yes 40/29 ... yes 38/30 ... yes 48/34 ... yes 53/36 ... yes 98/38 ... yes 40/39 ... yes 42/41 ... yes 40/42 ... yes 92/44 ... yes 115/46 ... yes 86/47 ... yes 115/48 ... yes 42/51 ... yes 15/53 ... yes 122/55 ... yes 115/57 ... yes 5/60 ... yes 44/62 ... yes 75/63 ... yes 128/64 ... yes 130/65 ... yes 8/66 ... yes 132/67 ... yes 68/68 ... yes 86/69 ... yes 116/70 ... yes 129/71 ... yes 138/72 ... yes 139/74 ... yes 31/75 ... yes 150/76 ... yes 138/78 ... yes 144/80 ... yes 167/81 ... yes 139/82 ... yes 172/84 ... yes 145/85 ... yes 103/89 ... yes 32/91 ... yes 184/93 ... yes 121/94 ... yes 106/96 ... yes 15/98 ... yes 184/99 ... yes 199/100 ... yes 15/104 ... yes 118/106 ... yes 98/107 ... yes 208/108 ... yes 40/109 ... yes 200/110 ... yes 32/111 ... yes 144/112 ... yes 180/113 ... yes 138/115 ... yes 163/118 ... yes 221/119 ... yes 221/120 ... yes 221/121 ... yes 221/122 ... yes 224/123 ... yes 224/124 ... yes 86/125 ... yes 144/127 ... yes 145/128 ... yes 145/129 ... yes 221/130 ... yes 241/131 ... yes 241/132 ... yes 242/133 ... yes 244/134 ... yes 245/136 ... yes 31/137 ... yes 31/138 ... yes 31/139 ... yes 31/140 ... yes 31/141 ... yes 31/142 ... yes 163/145 ... yes 31/146 ... yes 391/147 ... yes 53/149 ... yes 53/150 ... yes 144/151 ... yes 245/152 ... yes 145/153 ... yes 258/154 ... yes 106/155 ... yes 144/156 ... yes 42/157 ... yes 132/158 ... yes 145/159 ... yes 257/160 ... yes 144/161 ... yes 145/162 ... yes 53/163 ... yes 15/164 ... yes 53/165 ... yes 163/166 ... yes 111/167 ... yes 30/170 ... yes 42/171 ... yes 50/172 ... yes 252/173 ... yes 271/176 ... yes 271/177 ... yes 271/178 ... yes 50/180 ... yes 21/181 ... yes 270/182 ... yes 221/183 ... yes 21/184 ... yes 86/185 ... yes 98/186 ... yes 271/187 ... yes 271/188 ... yes 271/189 ... yes 72/191 ... yes 271/192 ... yes 271/193 ... yes 271/194 ... yes 271/195 ... yes 252/196 ... yes 223/197 ... yes 248/198 ... yes 262/201 ... yes 111/202 ... yes 206/203 ... yes 391/204 ... yes 189/205 ... yes 270/206 ... yes 184/207 ... yes 279/208 ... yes 332/209 ... yes 113/210 ... yes 258/211 ... yes 189/212 ... yes 138/214 ... yes 31/215 ... yes 113/217 ... yes 53/218 ... yes 319/221 ... yes 77/225 ... yes 53/226 ... yes 318/227 ... yes 31/228 ... yes 31/229 ... yes 31/231 ... yes 321/233 ... yes 321/234 ... yes 138/235 ... yes 50/236 ... yes 31/237 ... yes 144/238 ... yes 128/239 ... yes 189/241 ... yes 391/242 ... yes 35/243 ... yes 31/244 ... yes 113/245 ... yes 336/250 ... yes 392/251 ... yes 206/252 ... yes 28/253 ... yes 336/255 ... yes 336/259 ... yes 336/260 ... yes 336/261 ... yes 336/262 ... yes 336/263 ... yes 336/264 ... yes 336/265 ... yes 336/266 ... yes 336/267 ... yes 336/268 ... yes 336/269 ... yes 336/270 ... yes 336/271 ... yes 336/272 ... yes 336/273 ... yes 336/274 ... yes 336/275 ... yes 349/276 ... yes 53/279 ... yes 206/280 ... yes 189/281 ... yes 184/283 ... yes 346/284 ... yes 78/285 ... yes 46/286 ... yes 346/287 ... yes 346/288 ... yes 44/289 ... yes 106/290 ... yes 374/291 ... yes 368/292 ... yes 53/293 ... yes 346/294 ... yes 206/295 ... yes 368/296 ... yes 206/297 ... yes 389/298 ... yes 113/299 ... yes 189/301 ... yes 336/302 ... yes 491/303 ... yes 391/304 ... yes 32/305 ... yes 206/306 ... yes 407/309 ... yes 391/310 ... yes 189/311 ... yes 31/317 ... yes 189/318 ... yes 368/319 ... yes 221/320 ... yes 416/321 ... yes 189/322 ... yes 416/323 ... yes 148/324 ... yes 28/325 ... yes 40/326 ... yes 86/327 ... yes 262/328 ... yes 389/329 ... yes 50/330 ... yes 77/331 ... yes 416/332 ... yes 391/333 ... yes 442/334 ... yes 368/335 ... yes 263/336 ... yes 389/337 ... yes 418/338 ... yes 31/339 ... yes 368/340 ... yes 263/341 ... yes 31/342 ... yes 31/343 ... yes 449/344 ... yes 437/345 ... yes 53/346 ... yes 424/347 ... yes 449/348 ... yes 452/349 ... yes 263/350 ... yes 263/351 ... yes 346/352 ... yes 53/353 ... yes 53/354 ... yes 262/355 ... yes 442/356 ... yes 391/357 ... yes 346/358 ... yes 263/359 ... yes 392/360 ... yes 465/361 ... yes 465/362 ... yes 189/363 ... yes 189/364 ... yes 467/366 ... yes 468/367 ... yes 189/368 ... yes 416/369 ... yes 416/370 ... yes 138/371 ... yes 271/372 ... yes 271/373 ... yes 271/374 ... yes 271/375 ... yes 271/376 ... yes 53/377 ... yes 271/379 ... yes 113/380 ... yes 416/381 ... yes 416/382 ... yes 271/383 ... yes 416/384 ... yes 77/385 ... yes 32/386 ... yes 452/387 ... yes 181/388 ... yes 206/389 ... yes 596/390 ... yes 153/392 ... yes 31/393 ... yes 180/394 ... yes 360/395 ... yes 42/396 ... yes 138/398 ... yes 50/399 ... yes 416/404 ... yes 42/405 ... yes 416/406 ... yes 42/424 ... yes 31/426 ... yes 369/430 ... yes 420/431 ... yes 31/433 ... yes 31/434 ... yes 31/435 ... yes 509/436 ... yes 117/458 ... yes 514/459 ... yes 259/460 ... yes 376/461 ... yes 257/462 ... yes 43/463 ... yes 332/464 ... yes 507/465 ... yes 40/466 ... yes 420/468 ... yes 420/469 ... yes 184/470 ... yes 526/471 ... yes 262/472 ... yes 262/474 ... yes 527/475 ... yes 262/476 ... yes 528/477 ... yes 408/478 ... yes 31/479 ... yes 529/480 ... yes 530/481 ... yes 534/482 ... yes 533/483 ... yes 535/484 ... yes 536/485 ... yes 532/486 ... yes 43/487 ... yes 43/488 ... yes 531/489 ... yes 539/490 ... yes 541/491 ... yes 542/492 ... yes 262/493 ... yes 523/494 ... yes 46/495 ... yes 547/496 ... yes 547/497 ... yes 107/498 ... yes 86/499 ... yes 86/500 ... yes 290/501 ... yes 564/502 ... yes 32/503 ... yes 442/504 ... yes 139/506 ... yes 189/507 ... yes 583/511 ... yes 583/512 ... yes 583/513 ... yes 416/514 ... yes 583/516 ... yes 583/518 ... yes 53/519 ... yes 53/520 ... yes 583/521 ... yes 262/522 ... yes 583/523 ... yes 579/525 ... yes 596/526 ... yes 579/527 ... yes 546/528 ... yes 50/529 ... yes 599/530 ... yes 599/531 ... yes 599/532 ... yes 599/533 ... yes 321/534 ... yes 78/535 ... yes 416/536 ... yes 610/537 ... yes 599/538 ... yes 610/539 ... yes 271/540 ... yes 101/542 ... yes 614/543 ... yes 614/544 ... yes 599/547 ... yes 262/548 ... yes 583/550 ... yes 271/551 ... yes 599/554 ... yes 15/555 ... yes 491/556 ... yes 599/557 ... yes 262/558 ... yes 15/559 ... yes 262/560 ... yes 262/561 ... yes 583/562 ... yes 263/563 ... yes 262/564 ... yes 391/565 ... yes 626/566 ... yes 77/567 ... yes 583/568 ... yes 589/569 ... yes 583/570 ... yes 321/571 ... yes 601/572 ... yes 46/573 ... yes 42/574 ... yes 91/575 ... yes 189/576 ... yes 647/577 ... yes 50/579 ... yes 189/580 ... yes 189/581 ... yes 321/582 ... yes 321/583 ... yes 106/585 ... yes 106/588 ... yes 583/589 ... yes 583/590 ... yes 583/591 ... yes Redis version >= 5.0.0? ... yes Ruby version >= 2.7.2 ? ... yes (2.7.2) Git version >= 2.31.0 ? ... yes (2.32.0) Git user has default SSH configuration? ... yes Active users: ... 575 Is authorized keys file accessible? ... yes GitLab configured to store new projects in hashed storage? ... yes All projects are in hashed storage? ... yes Elasticsearch version 7.x (6.4 - 6.x deprecated to be removed in 13.8)? ... skipped (elasticsearch is disabled)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Possible fixes
Issue occurs at the following file, line 57
and 61
/opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20210713042153_finalize_ci_sources_pipelines_bigint_conversion.rb