DB-Migration failure while GitLab Upgrade from 17.1 or lower to 17.2 or higher if any user has "Validate user account" setting enabled

Summary

While upgrading GitLab from 17.1.x to 17.2.x, upgradation failed only if any Gitlab user enabled the "Validate user account" setting.

After disabling that setting from the user account's edit option, migration ran, and the GitLab Upgrade process was done.

Steps to reproduce

At least one GitLab user with the "Validate user account" checkbox enabled under the "Access" section of the Edit User Account page. In our case, the user had an "Administrator" Access Level.
Upgrade GitLab by executing the command docker compose up -d after changing the version from image: gitlab/gitlab-ce:17.1.3-ce.0 to image: gitlab/gitlab-ce:17.2.0-ce.0 inside docker-compose.yml file.

What is the current bug behavior?

DB Migration failed with an error and upgradation stopped.

What is the expected correct behavior?

DB Migration should work without any errors.

Relevant logs


Recipe: gitlab::database_migrations \* ruby_block\[check remote PG version\] action nothing (skipped due to action :nothing) \* rails_migration\[gitlab-rails\] action run\[2024-08-03T06:29:54+00:00\] WARN: gitlab-rails does not have a log_group or default logdir mode defined. Setting to 0700.

```
* bash_hide_env[migrate gitlab-rails database] action run
  [execute] rake aborted!
            StandardError: An error has occurred, all later migrations canceled:
            
            undefined method `last_digits' for #\u003cGitlab::BackgroundMigration::ConvertCreditCardValidationDataToHashes::CreditCardValidation user_id: 34, credit_card_validated_at: \"2022-09-24 11:23:58.217755000 +0000\", last_digits_hash: nil, holder_name_hash: nil, expiration_date_hash: nil, network_hash: nil, zuora_payment_method_xid: nil, stripe_setup_intent_xid: nil, stripe_payment_method_xid: nil, stripe_card_fingerprint: nil\u003e
            Did you mean?  last_digits_hash
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:23:in `block (2 levels) in perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `map'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `block in perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:105:in `block (2 levels) in each_sub_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batch_metrics.rb:22:in `instrument_operation'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:104:in `block in each_sub_batch'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block (2 levels) in each_batch'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block in each_batch'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `step'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `each_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:103:in `each_sub_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:16:in `perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:81:in `execute_batched_migration_job'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:63:in `execute_job'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:50:in `execute_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:25:in `perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:30:in `run_migration_job'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:140:in `run_migration_while'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:80:in `finalize'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:10:in `finalize'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:150:in `block (2 levels) in finalize_batched_background_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzers/base.rb:39:in `with_suppressed'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:149:in `block in finalize_batched_background_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/reestablished_connection_stack.rb:21:in `with_restored_connection_stack'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:148:in `finalize_batched_background_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:226:in `ensure_batched_background_migration_is_finished'
            /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240630231802_finalize_convert_credit_card_validation_data_to_hashes.rb:11:in `up'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:40:in `within'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:15:in `block in with_advisory_lock_connection'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:12:in `with_advisory_lock_connection'
            /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:136:in `configure_database'
            /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:107:in `configure_pg_databases'
            /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:94:in `block (3 levels) in \u003ctop (required)\u003e'
            /opt/gitlab/embedded/bin/bundle:25:in `load'
            /opt/gitlab/embedded/bin/bundle:25:in `\u003cmain\u003e'
            
            Caused by:
            NoMethodError: undefined method `last_digits' for #\u003cGitlab::BackgroundMigration::ConvertCreditCardValidationDataToHashes::CreditCardValidation user_id: 34, credit_card_validated_at: \"2022-09-24 11:23:58.217755000 +0000\", last_digits_hash: nil, holder_name_hash: nil, expiration_date_hash: nil, network_hash: nil, zuora_payment_method_xid: nil, stripe_setup_intent_xid: nil, stripe_payment_method_xid: nil, stripe_card_fingerprint: nil\u003e
            Did you mean?  last_digits_hash
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:23:in `block (2 levels) in perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `map'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `block in perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:105:in `block (2 levels) in each_sub_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batch_metrics.rb:22:in `instrument_operation'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:104:in `block in each_sub_batch'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block (2 levels) in each_batch'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block in each_batch'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `step'
            /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `each_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:103:in `each_sub_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:16:in `perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:81:in `execute_batched_migration_job'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:63:in `execute_job'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:50:in `execute_batch'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:25:in `perform'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:30:in `run_migration_job'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:140:in `run_migration_while'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:80:in `finalize'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:10:in `finalize'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:150:in `block (2 levels) in finalize_batched_background_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzers/base.rb:39:in `with_suppressed'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:149:in `block in finalize_batched_background_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/reestablished_connection_stack.rb:21:in `with_restored_connection_stack'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:148:in `finalize_batched_background_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:226:in `ensure_batched_background_migration_is_finished'
            /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240630231802_finalize_convert_credit_card_validation_data_to_hashes.rb:11:in `up'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:40:in `within'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:15:in `block in with_advisory_lock_connection'
            /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:12:in `with_advisory_lock_connection'
            /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:136:in `configure_database'
            /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:107:in `configure_pg_databases'
            /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:94:in `block (3 levels) in \u003ctop (required)\u003e'
            /opt/gitlab/embedded/bin/bundle:25:in `load'
            /opt/gitlab/embedded/bin/bundle:25:in `\u003cmain\u003e'
            Tasks: TOP =\u003e db:migrate
            (See full trace by running task with --trace)
            Running db:migrate rake task
            main: == [advisory_lock_connection] object_id: 53260, pg_backend_pid: 607
            main: == 20240630231802 FinalizeConvertCreditCardValidationDataToHashes: migrating ==
            main: -- transaction_open?(nil)
            main:    -\u003e 0.0000s
            main: -- transaction_open?(nil)
            main:    -\u003e 0.0000s
            main: == [advisory_lock_connection] object_id: 53260, pg_backend_pid: 607
  
  ================================================================================
  Error executing action `run` on resource 'bash_hide_env[migrate gitlab-rails database]'
  ================================================================================
  
  Mixlib::ShellOut::ShellCommandFailed
  ------------------------------------
  Expected process to exit with [0], but received '1'
  ---- Begin output of \"bash\"  ----
  STDOUT: rake aborted!
  StandardError: An error has occurred, all later migrations canceled:
  
  undefined method `last_digits' for #\u003cGitlab::BackgroundMigration::ConvertCreditCardValidationDataToHashes::CreditCardValidation user_id: 34, credit_card_validated_at: \"2022-09-24 11:23:58.217755000 +0000\", last_digits_hash: nil, holder_name_hash: nil, expiration_date_hash: nil, network_hash: nil, zuora_payment_method_xid: nil, stripe_setup_intent_xid: nil, stripe_payment_method_xid: nil, stripe_card_fingerprint: nil\u003e
  Did you mean?  last_digits_hash
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:23:in `block (2 levels) in perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `map'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `block in perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:105:in `block (2 levels) in each_sub_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batch_metrics.rb:22:in `instrument_operation'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:104:in `block in each_sub_batch'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block (2 levels) in each_batch'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block in each_batch'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `step'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `each_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:103:in `each_sub_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:16:in `perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:81:in `execute_batched_migration_job'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:63:in `execute_job'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:50:in `execute_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:25:in `perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:30:in `run_migration_job'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:140:in `run_migration_while'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:80:in `finalize'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:10:in `finalize'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:150:in `block (2 levels) in finalize_batched_background_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzers/base.rb:39:in `with_suppressed'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:149:in `block in finalize_batched_background_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/reestablished_connection_stack.rb:21:in `with_restored_connection_stack'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:148:in `finalize_batched_background_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:226:in `ensure_batched_background_migration_is_finished'
  /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240630231802_finalize_convert_credit_card_validation_data_to_hashes.rb:11:in `up'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:40:in `within'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:15:in `block in with_advisory_lock_connection'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:12:in `with_advisory_lock_connection'
  /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:136:in `configure_database'
  /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:107:in `configure_pg_databases'
  /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:94:in `block (3 levels) in \u003ctop (required)\u003e'
  /opt/gitlab/embedded/bin/bundle:25:in `load'
  /opt/gitlab/embedded/bin/bundle:25:in `\u003cmain\u003e'
  
  Caused by:
  NoMethodError: undefined method `last_digits' for #\u003cGitlab::BackgroundMigration::ConvertCreditCardValidationDataToHashes::CreditCardValidation user_id: 34, credit_card_validated_at: \"2022-09-24 11:23:58.217755000 +0000\", last_digits_hash: nil, holder_name_hash: nil, expiration_date_hash: nil, network_hash: nil, zuora_payment_method_xid: nil, stripe_setup_intent_xid: nil, stripe_payment_method_xid: nil, stripe_card_fingerprint: nil\u003e
  Did you mean?  last_digits_hash
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:23:in `block (2 levels) in perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `map'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:19:in `block in perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:105:in `block (2 levels) in each_sub_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batch_metrics.rb:22:in `instrument_operation'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:104:in `block in each_sub_batch'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block (2 levels) in each_batch'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:99:in `block in each_batch'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `step'
  /opt/gitlab/embedded/service/gitlab-rails/app/models/concerns/each_batch.rb:69:in `each_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/batched_migration_job.rb:103:in `each_sub_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/background_migration/convert_credit_card_validation_data_to_hashes.rb:16:in `perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:81:in `execute_batched_migration_job'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:63:in `execute_job'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:50:in `execute_batch'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_wrapper.rb:25:in `perform'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:30:in `run_migration_job'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:140:in `run_migration_while'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:80:in `finalize'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/background_migration/batched_migration_runner.rb:10:in `finalize'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:150:in `block (2 levels) in finalize_batched_background_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzers/base.rb:39:in `with_suppressed'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:149:in `block in finalize_batched_background_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/reestablished_connection_stack.rb:21:in `with_restored_connection_stack'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:148:in `finalize_batched_background_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/batched_background_migration_helpers.rb:226:in `ensure_batched_background_migration_is_finished'
  /opt/gitlab/embedded/service/gitlab-rails/db/post_migrate/20240630231802_finalize_convert_credit_card_validation_data_to_hashes.rb:11:in `up'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:33:in `block in exec_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/query_analyzer.rb:40:in `within'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/restrict_gitlab_schema.rb:30:in `exec_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migration_helpers/automatic_lock_writes_on_tables.rb:21:in `exec_migration'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:54:in `ddl_transaction'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/runner_backoff/active_record_mixin.rb:21:in `execute_migration_in_transaction'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:15:in `block in with_advisory_lock_connection'
  /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/pg_backend_pid.rb:12:in `with_advisory_lock_connection'
  /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:136:in `configure_database'
  /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:107:in `configure_pg_databases'
  /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:94:in `block (3 levels) in \u003ctop (required)\u003e'
  /opt/gitlab/embedded/bin/bundle:25:in `load'
  /opt/gitlab/embedded/bin/bundle:25:in `\u003cmain\u003e'
  Tasks: TOP =\u003e db:migrate
  (See full trace by running task with --trace)
  Running db:migrate rake task
  main: == [advisory_lock_connection] object_id: 53260, pg_backend_pid: 607
  main: == 20240630231802 FinalizeConvertCreditCardValidationDataToHashes: migrating ==
  main: -- transaction_open?(nil)
  main:    -\u003e 0.0000s
  main: -- transaction_open?(nil)
  main:    -\u003e 0.0000s
  main: == [advisory_lock_connection] object_id: 53260, pg_backend_pid: 607
  STDERR: 
  ---- End output of \"bash\"  ----
  Ran \"bash\"  returned 1
  
  Cookbook Trace: (most recent call first)
  ----------------------------------------
  /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb:20:in `block in class_from_file'
  
  Resource Declaration:
  ---------------------
  # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb
  
   20:   bash_hide_env \"migrate #{new_resource.name} database\" do
   21:     code \u003c\u003c-EOH
   22:     set -e
   23:     log_file=\"#{logging_settings[:log_directory]}/#{new_resource.logfile_prefix}-$(date +%Y-%m-%d-%H-%M-%S).log\"
   24:     umask 077
   25:     /opt/gitlab/bin/gitlab-rake #{new_resource.rake_task} 2\u003e\u0026 1 | tee ${log_file}
   26:     STATUS=${PIPESTATUS[0]}
   27:     chown #{account_helper.gitlab_user}:#{account_helper.gitlab_group} ${log_file}
   28:     echo $STATUS \u003e #{new_resource.helper.db_migrate_status_file}
   29:     exit $STATUS
   30:     EOH
   31: 
   32:     environment new_resource.environment if new_resource.property_is_set?(:environment)
   33:     new_resource.dependent_services.each do |svc|
   34:       notifies :restart, svc, :immediately
   35:     end
   36: 
   37:     not_if { new_resource.helper.migrated? }
   38:     sensitive false
   39:   end
   40: end
  
  Compiled Resource:
  ------------------
  # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb:20:in `block in class_from_file'
  
  bash_hide_env(\"migrate gitlab-rails database\") do
    action [:run]
    default_guard_interpreter :default
    interpreter \"bash\"
    declared_type :bash_hide_env
    cookbook_name \"gitlab\"
    recipe_name \"database_migrations\"
    code \"    set -e\\n    log_file=\\\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\\\"\\n    umask 077\\n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2\u003e\u0026 1 | tee ${log_file}\\n    STATUS=${PIPESTATUS[0]}\\n    chown git:git ${log_file}\\n    echo $STATUS \u003e /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-c6697a9cd49e1cfa45cc31b1ecdb90ae-b30193cc04c\\n    exit $STATUS\\n\"
    environment \"*sensitive value suppressed*\"
    not_if { #code block }
  end
  
  System Info:
  ------------
  chef_version=18.3.0
  platform=ubuntu
  platform_version=22.04
  ruby=ruby 3.1.5p253 (2024-04-023 revision 1945f8dc0e) [x86_64-linux]
  program_name=/opt/gitlab/embedded/bin/cinc-client
  executable=/opt/gitlab/embedded/bin/cinc-client
```

Details of package version

Provide the package version installation details

The setup is using GitLab Docker Image (image: 'gitlab/gitlab-ce:17.2.1-ce.0') (https://hub.docker.com/layers/gitlab/gitlab-ce/17.2.1-ce.0/images/sha256-9a5dff8bec16a32551b2069de505be134b30024e022ee72f30d0eddb9cccac95)

Environment details

Containerized using docker with official GitLab docker Image. Mentioning the host server information below

Operating System: Red Hat Enterprise Linux release 8.10 (Ootpa)
Installation Target, remove incorrect values:
- Other: Docker Container over Dedicated Server
Installation Type, remove incorrect values:
- Upgrade from version 17.1.3
Is there any other software running on the machine: N/A
Is this a single or multiple node installation?: Single
Resources
- CPU: 72 Core (3800 MHz)
- Memory total: 36 GB
Platform: x86_64

Configuration details

Below are some of the configurations settled in the environment variable of the Docker Compose file.

gitlab:
  image: 'gitlab/gitlab-ce:17.1.2-ce.0'
  hostname: 'localhost'
  environment: GITLAB_OMNIBUS_CONFIG: |
    external_url 'DUMMY-VALUE'
    gitlab_rails['omniauth_providers'] = [
      {
        'name': 'gitlab',
        'app_id': 'DUMMY-VALUE',
        'app_secret': 'DUMMY-VALUE',
        'args': { 'scope': 'read_user' }
      }
    ]
    gitlab_rails['registry_enabled'] = true
    puma['worker_processes'] = 4
  shm_size: '1gb'
  deploy:
    resources:
      limits:
        memory: 36G
  ports:
    - 'DUMMY-VALUE:DUMMY-VALUE'
  volumes:
    - 'gitlab-data-config:/etc/gitlab'
    - 'gitlab-data-logs:/var/log/gitlab'
    - 'gitlab-data-data:/var/opt/gitlab'