Skip to content

DB Migrations fail on upgrade from 14.5.2 to 14.6

Summary

The upgrade from gitlab-ee 14.5.2 to 14.6 fails in the db migration step.

Steps to reproduce

Run the upgrade with apt upgrade

Example Project

What is the current bug behavior?

Currently the upgrade fails in the db migration step.

What is the expected correct behavior?

The upgrade should work and GitLab should be upgraded to 15.6

Relevant logs and/or screenshots

System Info:
    ------------
    chef_version=15.17.4
    platform=debian
    platform_version=10
    ruby=ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client
Running handlers:
There was an error running gitlab-ctl reconfigure:
rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash[migrate gitlab-rails database] (/opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/resources/rails_migration.rb line 16) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  "/tmp/chef-script20211228-26258-1pafjt2" ----
STDOUT: rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:
data must not be empty
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/crypto_helper.rb:20:in `aes256_gcm_encrypt'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211126115449_encrypt_static_objects_external_storage_auth_token.rb:16:in `block in up'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211126115449_encrypt_static_objects_external_storage_auth_token.rb:15:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:265:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:264:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:63:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
ArgumentError: data must not be empty
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/crypto_helper.rb:20:in `aes256_gcm_encrypt'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211126115449_encrypt_static_objects_external_storage_auth_token.rb:16:in `block in up'
/opt/gitlab/embedded/service/gitlab-rails/db/migrate/20211126115449_encrypt_static_objects_external_storage_auth_token.rb:15:in `up'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `block in write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:112:in `block in read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:172:in `retry_with_backoff'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:110:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:125:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:77:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:265:in `block in transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database.rb:264:in `transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/migrations/lock_retry_mixin.rb:31:in `ddl_transaction'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:63:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)
== 20211126115449 EncryptStaticObjectsExternalStorageAuthToken: migrating =====
STDERR:
---- End output of "bash"  "/tmp/chef-script20211228-26258-1pafjt2" ----
Ran "bash"  "/tmp/chef-script20211228-26258-1pafjt2" returned 1
Running handlers complete
Chef Infra Client failed. 2 resources updated in 22 seconds
===
There was an error running gitlab-ctl reconfigure. Please check the output above for more
details.
===
dpkg: error processing package gitlab-ee (--configure):
 installed gitlab-ee package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
 gitlab-ee
E: Sub-process /usr/bin/dpkg returned an error code (1)