Migration failed when updating from 15.4.6 CE to 15.10.2CE
I am getting the following error when I am trying to upgrade my instance from 15.4.6 CE to 15.10.2CE. I have already tried the following things
- Manually migrating the failed migration didn't work
- Tried to restore to older version from the backup and still getting the same error when running sudo gitlab-ctl reconfigure
Please find the trace below
[2023-04-12T20:32:21+00:00] ERROR: Running exception handlers There was an error running gitlab-ctl reconfigure:
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 18) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1' ---- Begin output of "bash" ---- STDOUT: rake aborted! StandardError: An error has occurred, all later migrations canceled:
PG::CheckViolation: ERROR: check constraint "check_70f294ef54" is violated by some row /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:119:in
block in validate_check_constraint' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/timeout_helpers.rb:31:indisable_statement_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:116:invalidate_check_constraint' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:232:invalidate_not_null_constraint' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20230223014251_validate_not_null_constraint_on_oauth_access_tokens_expires_in.rb:7:inup' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:inblock in exec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:37:inwithin' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:inexec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:19:inexec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:inddl_transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:115:inconfigure_database' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:95:inblock (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:inload' /opt/gitlab/embedded/bin/bundle:23:in'Caused by: ActiveRecord::StatementInvalid: PG::CheckViolation: ERROR: check constraint "check_70f294ef54" is violated by some row /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:119:in
block in validate_check_constraint' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/timeout_helpers.rb:31:indisable_statement_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:116:invalidate_check_constraint' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:232:invalidate_not_null_constraint' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20230223014251_validate_not_null_constraint_on_oauth_access_tokens_expires_in.rb:7:inup' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:inblock in exec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:37:inwithin' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:inexec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:19:inexec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:inddl_transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:115:inconfigure_database' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:95:inblock (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:inload' /opt/gitlab/embedded/bin/bundle:23:in'Caused by: PG::CheckViolation: ERROR: check constraint "check_70f294ef54" is violated by some row /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:119:in
block in validate_check_constraint' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/timeout_helpers.rb:31:indisable_statement_timeout' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:116:invalidate_check_constraint' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/constraints_helpers.rb:232:invalidate_not_null_constraint' /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20230223014251_validate_not_null_constraint_on_oauth_access_tokens_expires_in.rb:7:inup' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:inblock in exec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:37:inwithin' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:inexec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:19:inexec_migration' /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:36:inddl_transaction' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:115:inconfigure_database' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:95:inblock (3 levels) in <top (required)>' /opt/gitlab/embedded/bin/bundle:23:inload' /opt/gitlab/embedded/bin/bundle:23:in' Tasks: TOP => db:migrate (See full trace by running task with --trace) main: == 20230223014251 ValidateNotNullConstraintOnOauthAccessTokensExpiresIn: migrating main: -- execute("SET statement_timeout TO 0") main: -> 0.0005s main: -- execute("ALTER TABLE oauth_access_tokens VALIDATE CONSTRAINT check_70f294ef54;") main: -- execute("RESET statement_timeout") main: -> 0.0004s STDERR: ---- End output of "bash" ---- Ran "bash" returned 1