Migration errorr on gitlab.com during 11.5.0-rc1.ee.0 deployment
https://ops.gitlab.net/gitlab-org/takeoff/-/jobs/19909
During the 11.5.0-rc1 deployment we see a ForeignKeyViolation:
2018-11-06 10:11:38.164866 I [17:main] ProgressRunner -- 10.218.3.2 -> 71.2699s
10.218.3.2 -- execute("RESET ALL")
10.218.3.2 -> 0.0014s
10.218.3.2 -- transaction_open?()
10.218.3.2 -> 0.0000s
10.218.3.2 -- foreign_keys(:geo_event_log)
10.218.3.2 -> 0.0042s
10.218.3.2 -- execute("ALTER TABLE geo_event_log\nADD CONSTRAINT fk_304067fc30\nFOREIGN KEY (hashed_storage_attachments_event_id)\nREFERENCES geo_hashed_storage_attachments_events (id)\nON DELETE CASCADE\nNOT VALID;\n")
10.218.3.2 -> 0.0040s
10.218.3.2 -- execute("SET statement_timeout TO 0")
10.218.3.2 -> 0.0005s
10.218.3.2 -- execute("ALTER TABLE geo_event_log VALIDATE CONSTRAINT fk_304067fc30;")
10.218.3.2 -- execute("RESET ALL")
10.218.3.2 -> 0.0006s
2018-11-06 10:11:38.366713 I [17:main] ProgressRunner -- 10.218.3.2 rake aborted!
10.218.3.2 StandardError: An error has occurred, all later migrations canceled:
10.218.3.2
10.218.3.2 PG::ForeignKeyViolation: ERROR: insert or update on table "geo_event_log" violates foreign key constraint "fk_304067fc30"
10.218.3.2 DETAIL: Key (hashed_storage_attachments_event_id)=(210) is not present in table "geo_hashed_storage_attachments_events".
10.218.3.2 : ALTER TABLE geo_event_log VALIDATE CONSTRAINT fk_304067fc30;
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:204:in `block in add_concurrent_foreign_key'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in `disable_statement_timeout'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:203:in `add_concurrent_foreign_key'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:41:in `block in up'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:40:in `each'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:40:in `up'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `load'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `<main>'
10.218.3.2
10.218.3.2 Caused by:
10.218.3.2 ActiveRecord::InvalidForeignKey: PG::ForeignKeyViolation: ERROR: insert or update on table "geo_event_log" violates foreign key constraint "fk_304067fc30"
10.218.3.2 DETAIL: Key (hashed_storage_attachments_event_id)=(210) is not present in table "geo_hashed_storage_attachments_events".
10.218.3.2 : ALTER TABLE geo_event_log VALIDATE CONSTRAINT fk_304067fc30;
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:204:in `block in add_concurrent_foreign_key'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in `disable_statement_timeout'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:203:in `add_concurrent_foreign_key'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:41:in `block in up'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:40:in `each'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:40:in `up'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `load'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `<main>'
10.218.3.2
10.218.3.2 Caused by:
10.218.3.2 PG::ForeignKeyViolation: ERROR: insert or update on table "geo_event_log" violates foreign key constraint "fk_304067fc30"
10.218.3.2 DETAIL: Key (hashed_storage_attachments_event_id)=(210) is not present in table "geo_hashed_storage_attachments_events".
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:204:in `block in add_concurrent_foreign_key'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:254:in `disable_statement_timeout'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers.rb:203:in `add_concurrent_foreign_key'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:41:in `block in up'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:40:in `each'
10.218.3.2 /opt/gitlab/embedded/service/gitlab-rails/ee/db/migrate/20181017131623_add_missing_geo_even_log_indexes.rb:40:in `up'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `load'
10.218.3.2 /opt/gitlab/embedded/bin/bundle:23:in `<main>'
10.218.3.2 Tasks: TOP => db:migrate
10.218.3.2 (See full trace by running task with --trace)