Docker version GitLab 18.3/18.4 crash when restart if registry database enabled
Summary
I had a self-host Docker version GitLab CE with registry metadata database enabled.
After upgrade to 18.3, it always crash when containter restart.
Still have the same issue in 18.4.
Workaround is delete the containter and create a new one(with existed data).
Steps to reproduce
Example GitLab configurate (gitlab.rb):
external_url 'http://MyGitLab'
letsencrypt['enable'] = false
registry_external_url 'http://MyGitLab:5100'
registry['database'] = {
'enabled' => true,
}
Step:
-
Create a fold(
$GITLAB_DIR) to store data of GitLab, include subfolders:config,data,dump -
copy
gitlab.rbinto$GITLAB_DIR/config -
Create a new GitLab containter:
docker run -d --name gitlab \ -p 80:80 -p 5100:5100 \ -v $GITLAB_DIR/dump:/opt/gitlab/embedded/cookbooks/cache/ \ -v $GITLAB_DIR/config:/etc/gitlab \ -v $GITLAB_DIR/data:/var/opt/gitlab \ gitlab/gitlab-ce:latest -
Wait GitLab initialized, signin and do something(optional)
-
stop the containter
docker stop gitlab -
Start the GitLab containter again and fail
docker start gitlab
What is the current bug behavior?
Gitlab crash and can't restart.
What is the expected correct behavior?
GitLab Service work
Relevant logs and/or screenshots
The error message:
[2025-09-24T07:39:46+00:00] ERROR: Running exception handlers
There was an error running gitlab-ctl reconfigure:
registry_database_migrations[registry] (registry::database_migrations line 19) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate registry database: registry] (registry::database_migrations line 15) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" ----
STDOUT: Running migrate up
Executing command:
/opt/gitlab/embedded/bin/registry database migrate up /var/opt/gitlab/registry/config.yml
time="2025-09-24T07:39:46Z" level=error msg=Connect database=registry duration_ms=2 err="failed to connect to `user=registry database=registry`: /var/opt/gitlab/postgresql/.s.PGSQL.5432 (/var/opt/gitlab/postgresql): server error: FATAL: the database system is starting up (SQLSTATE 57P03)" host=/var/opt/gitlab/postgresql port=5432
failed to construct database connection: verification failed: failed to connect to `user=registry database=registry`: /var/opt/gitlab/postgresql/.s.PGSQL.5432 (/var/opt/gitlab/postgresql): server error: FATAL: the database system is starting up (SQLSTATE 57P03)
STDERR:
---- End output of "bash" ----
Ran "bash" returned 1
Running handlers complete
[2025-09-24T07:39:46+00:00] ERROR: Exception handlers complete
Infra Phase failed. 22 resources updated in 09 seconds
[2025-09-24T07:39:46+00:00] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
[2025-09-24T07:39:46+00:00] FATAL: ---------------------------------------------------------------------------------------
[2025-09-24T07:39:46+00:00] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
[2025-09-24T07:39:46+00:00] FATAL: ---------------------------------------------------------------------------------------
[2025-09-24T07:39:46+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: registry_database_migrations[registry] (registry::database_migrations line 19) had an error: Mixlib::ShellOut::ShellCommandFailed: bash_hide_env[migrate registry database: registry] (registry::database_migrations line 15) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" ----
STDOUT: Running migrate up
Executing command:
/opt/gitlab/embedded/bin/registry database migrate up /var/opt/gitlab/registry/config.yml
time="2025-09-24T07:39:46Z" level=error msg=Connect database=registry duration_ms=2 err="failed to connect to `user=registry database=registry`: /var/opt/gitlab/postgresql/.s.PGSQL.5432 (/var/opt/gitlab/postgresql): server error: FATAL: the database system is starting up (SQLSTATE 57P03)" host=/var/opt/gitlab/postgresql port=5432
failed to construct database connection: verification failed: failed to connect to `user=registry database=registry`: /var/opt/gitlab/postgresql/.s.PGSQL.5432 (/var/opt/gitlab/postgresql): server error: FATAL: the database system is starting up (SQLSTATE 57P03)
STDERR:
---- End output of "bash" ----
Ran "bash" returned 1
Dump file: cinc-stacktrace.out
Results of GitLab environment info
Expand for output related to GitLab environment info
System information\ System:\ Current User: git\ Using RVM: no\ Ruby Version: 3.2.8\ Gem Version: 3.7.1\ Bundler Version:2.7.1\ Rake Version: 13.0.6\ Redis Version: 7.2.10\ Sidekiq Version:7.3.9\ Go Version: unknown\ \ GitLab information\ Version: 18.4.0\ Revision: 01f1e46b662\ Directory: /opt/gitlab/embedded/service/gitlab-rails\ DB Adapter: PostgreSQL\ DB Version: 16.8\ URL: http://gaea3.nchc.org.tw\ HTTP Clone URL: http://gaea3.nchc.org.tw/some-group/some-project.git\ SSH Clone URL: git@gaea3.nchc.org.tw:some-group/some-project.git\ Using LDAP: no\ Using Omniauth: yes\ Omniauth Providers:\ \ GitLab Shell\ Version: 14.45.2\ Repository storages:\ - default: unix:/var/opt/gitlab/gitaly/gitaly.socket\ GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell\ \ Gitaly\ - default Address: unix:/var/opt/gitlab/gitaly/gitaly.socket\ - default Version: 18.4.0\ - default Git Version: 2.50.1
Edited by 🤖 GitLab Bot 🤖