gitlab-ce docker installation upgrade 17.11.4 to 18.1.0 fails
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
I have an installation of the gitlab-ce docker image on Ubuntu 22.04 with a pretty basic setup. It successfully upgraded for the past years and is now at v17.11.4. Trying to upgrade to v18.1.0 fails in a restart loop with the error in the log below. The older v17.11.4 image runs fine.
Log output:
{"log":"---- End output of \"bash\" ----\n","stream":"stderr","time":"2025-06-21T23:00:42.86276721Z"}
{"log":"Ran \"bash\" returned 1\n","stream":"stderr","time":"2025-06-21T23:00:42.862770605Z"}
{"log":"\n","stream":"stderr","time":"2025-06-21T23:00:42.8627739Z"}
{"log":"Running handlers complete\n","stream":"stdout","time":"2025-06-21T23:00:42.863057934Z"}
{"log":"[2025-06-21T23:00:42+00:00] ERROR: Exception handlers complete\n","stream":"stdout","time":"2025-06-21T23:00:42.863140618Z"}
{"log":"Infra Phase failed. 14 resources updated in 37 seconds\n","stream":"stdout","time":"2025-06-21T23:00:42.863524137Z"}
{"log":"[2025-06-21T23:00:42+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out\n","stream":"stdout","time":"2025-06-21T23:00:42.865567608Z"}
{"log":"[2025-06-21T23:00:42+00:00] FATAL: ---------------------------------------------------------------------------------------\n","stream":"stdout","time":"2025-06-21T23:00:42.865578569Z"}
{"log":"[2025-06-21T23:00:42+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT\n","stream":"stdout","time":"2025-06-21T23:00:42.865582409Z"}
{"log":"[2025-06-21T23:00:42+00:00] FATAL: ---------------------------------------------------------------------------------------\n","stream":"stdout","time":"2025-06-21T23:00:42.865766586Z"}
{"log":"[2025-06-21T23:00:42+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'\n","stream":"stdout","time":"2025-06-21T23:00:42.866167927Z"}
{"log":"---- Begin output of \"bash\" ----\n","stream":"stdout","time":"2025-06-21T23:00:42.86618556Z"}
{"log":"STDOUT: rake aborted!\n","stream":"stdout","time":"2025-06-21T23:00:42.866189325Z"}
{"log":"ActiveRecord::DatabaseConnectionError: There is an issue connecting to your database with your username/password, username: gitlab.\n","stream":"stdout","time":"2025-06-21T23:00:42.866192546Z"}
{"log":"\n","stream":"stdout","time":"2025-06-21T23:00:42.866195915Z"}
{"log":"Please check your database configuration to ensure the username/password are valid.\n","stream":"stdout","time":"2025-06-21T23:00:42.86619917Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:32:in `block in reconnect!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866202603Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:31:in `reconnect!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866206106Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:89:in `block in verify!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866222349Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:80:in `verify!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866226636Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:126:in `read_write'\n","stream":"stdout","time":"2025-06-21T23:00:42.866230061Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'\n","stream":"stdout","time":"2025-06-21T23:00:42.866233348Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'\n","stream":"stdout","time":"2025-06-21T23:00:42.866236894Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:179:in `configure_database'\n","stream":"stdout","time":"2025-06-21T23:00:42.866240419Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:156:in `configure_pg_databases'\n","stream":"stdout","time":"2025-06-21T23:00:42.866243735Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:102:in `block (3 levels) in \u003ctop (required)\u003e'\n","stream":"stdout","time":"2025-06-21T23:00:42.866247015Z"}
{"log":"/opt/gitlab/embedded/bin/bundle:25:in `load'\n","stream":"stdout","time":"2025-06-21T23:00:42.866250758Z"}
{"log":"/opt/gitlab/embedded/bin/bundle:25:in `\u003cmain\u003e'\n","stream":"stdout","time":"2025-06-21T23:00:42.866254015Z"}
{"log":"\n","stream":"stdout","time":"2025-06-21T23:00:42.866257359Z"}
{"log":"Caused by:\n","stream":"stdout","time":"2025-06-21T23:00:42.86626043Z"}
{"log":"PG::ConnectionBad: connection to server on socket \"/var/opt/gitlab/postgresql/.s.PGSQL.5432\" failed: Connection refused\n","stream":"stdout","time":"2025-06-21T23:00:42.866264359Z"}
{"log":"\u0009Is the server running locally and accepting connections on that socket?\n","stream":"stdout","time":"2025-06-21T23:00:42.866268135Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:32:in `block in reconnect!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866271599Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:31:in `reconnect!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866275024Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:89:in `block in verify!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866278353Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:80:in `verify!'\n","stream":"stdout","time":"2025-06-21T23:00:42.866301279Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:126:in `read_write'\n","stream":"stdout","time":"2025-06-21T23:00:42.866305472Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'\n","stream":"stdout","time":"2025-06-21T23:00:42.866308882Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'\n","stream":"stdout","time":"2025-06-21T23:00:42.866312703Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:179:in `configure_database'\n","stream":"stdout","time":"2025-06-21T23:00:42.866320739Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:156:in `configure_pg_databases'\n","stream":"stdout","time":"2025-06-21T23:00:42.866324141Z"}
{"log":"/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:102:in `block (3 levels) in \u003ctop (required)\u003e'\n","stream":"stdout","time":"2025-06-21T23:00:42.866327464Z"}
{"log":"/opt/gitlab/embedded/bin/bundle:25:in `load'\n","stream":"stdout","time":"2025-06-21T23:00:42.866330872Z"}
{"log":"/opt/gitlab/embedded/bin/bundle:25:in `\u003cmain\u003e'\n","stream":"stdout","time":"2025-06-21T23:00:42.866334204Z"}
{"log":"Tasks: TOP =\u003e gitlab:db:configure\n","stream":"stdout","time":"2025-06-21T23:00:42.866337544Z"}
{"log":"(See full trace by running task with --trace)\n","stream":"stdout","time":"2025-06-21T23:00:42.866340863Z"}
{"log":"STDERR: \n","stream":"stdout","time":"2025-06-21T23:00:42.866344199Z"}
{"log":"---- End output of \"bash\" ----\n","stream":"stdout","time":"2025-06-21T23:00:42.866347311Z"}Output of /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out:
Generated at 2025-06-21 22:50:09 +0000
Mixlib::ShellOut::ShellCommandFailed: rails_migration[gitlab-rails] (gitlab::database_migrations line 51) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate gitlab-rails database] (gitlab::database_migrations line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" ----
STDOUT: rake aborted!
ActiveRecord::DatabaseConnectionError: There is an issue connecting to your database with your username/password, username: gitlab.
Please check your database configuration to ensure the username/password are valid.
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:32:in `block in reconnect!'
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:31:in `reconnect!'
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:89:in `block in verify!'
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:80:in `verify!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:126:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:179:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:156:in `configure_pg_databases'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:102:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Caused by:
PG::ConnectionBad: connection to server on socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432" failed: Connection refused
Is the server running locally and accepting connections on that socket?
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:32:in `block in reconnect!'
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:31:in `reconnect!'
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:89:in `block in verify!'
/opt/gitlab/embedded/service/gitlab-rails/gems/activerecord-gitlab/lib/active_record/gitlab_patches/abstract_adapter.rb:80:in `verify!'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/load_balancer.rb:126:in `read_write'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:126:in `write_using_load_balancer'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/database/load_balancing/connection_proxy.rb:96:in `method_missing'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:179:in `configure_database'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:156:in `configure_pg_databases'
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:102:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:25:in `load'
/opt/gitlab/embedded/bin/bundle:25:in `<main>'
Tasks: TOP => gitlab:db:configure
(See full trace by running task with --trace)Edited by 🤖 GitLab Bot 🤖