Migration error on deploy 11.5.0-rc1.ee.0 deployment
When db/migrate/20181031190559_drop_gcp_clusters_table.rb
ran on procution database we got the following error:
2018-11-06 10:45:32.044828 I [19:main] ProgressRunner -- 10.218.3.2 -> 0.0112s
10.218.3.2 -- transaction_open?()
10.218.3.2 -> 0.0000s
10.218.3.2 -- index_exists?(:geo_event_log, :upload_deleted_event_id, {:where=>"upload_deleted_event_id IS NOT NULL", :algorithm=>:concurrently})
10.218.3.2 -> 0.0114s
10.218.3.2 -- transaction_open?()
10.218.3.2 -> 0.0000s
10.218.3.2 -- foreign_keys(:geo_event_log)
10.218.3.2 -> 0.0046s
10.218.3.2 -- execute("SET statement_timeout TO 0")
10.218.3.2 -> 0.0008s
10.218.3.2 -- execute("ALTER TABLE geo_event_log VALIDATE CONSTRAINT fk_304067fc30;")
10.218.3.2 -> 0.0035s
10.218.3.2 -- execute("RESET ALL")
10.218.3.2 -> 0.0008s
10.218.3.2 == 20181017131623 AddMissingGeoEvenLogIndexes: migrated (58.0476s) ============
10.218.3.2
10.218.3.2 == 20181031190559 DropGcpClustersTable: migrating =============================
10.218.3.2 -- drop_table(:gcp_clusters)
2018-11-06 10:45:37.464518 I [19:main] ProgressRunner -- 10.218.3.2 rake aborted!
10.218.3.2 StandardError: An error has occurred, this and all later migrations canceled:
10.218.3.2
10.218.3.2 PG::TRDeadlockDetected: ERROR: deadlock detected
10.218.3.2 DETAIL: Process 17407 waits for AccessExclusiveLock on relation 33909 of database 16385; blocked by process 6503.
10.218.3.2 Process 6503 waits for AccessShareLock on relation 33706 of database 16385; blocked by process 17407.
10.218.3.2 HINT: See server log for query details.
10.218.3.2 : DROP TABLE "gcp_clusters"
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20181031190559_drop_gcp_clusters_table.rb:9:in `up'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `load'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `<main>'
10.218.3.2
10.218.3.2 Caused by:
10.218.3.2 ActiveRecord::StatementInvalid: PG::TRDeadlockDetected: ERROR: deadlock detected
10.218.3.2 DETAIL: Process 17407 waits for AccessExclusiveLock on relation 33909 of database 16385; blocked by process 6503.
10.218.3.2 Process 6503 waits for AccessShareLock on relation 33706 of database 16385; blocked by process 17407.
10.218.3.2 HINT: See server log for query details.
10.218.3.2 : DROP TABLE "gcp_clusters"
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20181031190559_drop_gcp_clusters_table.rb:9:in `up'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `load'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `<main>'
10.218.3.2
10.218.3.2 Caused by:
10.218.3.2 PG::TRDeadlockDetected: ERROR: deadlock detected
10.218.3.2 DETAIL: Process 17407 waits for AccessExclusiveLock on relation 33909 of database 16385; blocked by process 6503.
10.218.3.2 Process 6503 waits for AccessShareLock on relation 33706 of database 16385; blocked by process 17407.
10.218.3.2 HINT: See server log for query details.
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/db/migrate/20181031190559_drop_gcp_clusters_table.rb:9:in `up'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `load'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `<main>'
10.218.3.2 Tasks: TOP => db:migrate
10.218.3.2 (See full trace by running task with --trace)
Full log of deployment https://ops.gitlab.net/gitlab-org/takeoff/-/jobs/19933
MR of adding the migration in the first place https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/22713
Edited by Steve Xuereb