60 seconds hang at "WARN: Connecting to GitLab to generate new app_id and app_secret for GitLab Mattermost."

Summary

When running sudo gitlab-ctl reconfigure, there always is a 60 second delay at ruby_block[authorize mattermost with gitlab]

Steps to reproduce

Don't know exactly what I'm doing wrong. This installation is a couple years old and Mattermost integration has been active since almost the beginning.

Also - and I'm sorry for that - I know certainly that this problem already exists since a long time, I just never had gotten around to opening up this issue.

What is the current bug behavior?

Reconfiguring just hangs for exactly a minute and two WARN: messages are printed.

What is the expected correct behavior?

It shouldn't hang 😉

Relevant logs

Recipe: mattermost::enable
  * account[Mattermost user and group] action create
    * group[Mattermost user and group] action create (up to date)
    * linux_user[Mattermost user and group] action create (up to date)
     (up to date)
  * directory[/var/opt/gitlab/mattermost] action create (up to date)
  * directory[/var/log/gitlab/mattermost] action create (up to date)
  * directory[/fsapp/gitlab_data/mattermost_data] action create (up to date)
  * directory[/var/opt/gitlab/mattermost/plugins] action create (up to date)
  * directory[/var/opt/gitlab/mattermost/client-plugins] action create (up to date)
  * file[/var/log/gitlab/mattermost/mattermost.log] action create (up to date)
  * postgresql_user[gitlab_mattermost] action create
    * execute[create gitlab_mattermost postgresql user] action run (skipped due to not_if)
     (up to date)
  * execute[create mattermost_production database] action run (skipped due to not_if)
  * ruby_block[authorize mattermost with gitlab] action run[2023-02-23T09:55:09+01:00] WARN: Connecting to GitLab to generate new app_id and app_secret for GitLab Mattermost.
[2023-02-23T09:56:59+01:00] WARN: Something went wrong while trying to update gitlab-secrets.json. Check the file permissions and try reconfiguring again.
[2023-02-23T09:56:59+01:00] INFO: ruby_block[authorize mattermost with gitlab] called
    - execute the ruby block authorize mattermost with gitlab
  * ruby_block[populate mattermost configuration options] action run[2023-02-23T09:56:59+01:00] INFO: ruby_block[populate mattermost configuration options] called
    - execute the ruby block populate mattermost configuration options
  * remote_file[/var/opt/gitlab/mattermost/config.json] action create_if_missing (up to date)
  * env_dir[/opt/gitlab/etc/mattermost/env] action create
    * directory[/opt/gitlab/etc/mattermost/env] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SERVICESETTINGS_SITEURL] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SERVICESETTINGS_LISTENADDRESS] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SERVICESETTINGS_ALLOWEDUNTRUSTEDINTERNALCONNECTIONS] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SERVICESETTINGS_ENABLEAPITEAMDELETION] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_TEAMSETTINGS_SITENAME] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SQLSETTINGS_DRIVERNAME] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SQLSETTINGS_DATASOURCE] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SQLSETTINGS_ATRESTENCRYPTKEY] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_LOGSETTINGS_FILELOCATION] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_FILESETTINGS_DIRECTORY] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_GITLABSETTINGS_ENABLE] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_GITLABSETTINGS_SECRET] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_GITLABSETTINGS_ID] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_GITLABSETTINGS_SCOPE] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_GITLABSETTINGS_AUTHENDPOINT] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_GITLABSETTINGS_TOKENENDPOINT] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_GITLABSETTINGS_USERAPIENDPOINT] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_PLUGINSETTINGS_DIRECTORY] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_PLUGINSETTINGS_CLIENTDIRECTORY] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_INSTALL_TYPE] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/SSL_CERT_DIR] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_NOTIFICATIONLOGSETTINGS_FILELOCATION] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_SERVICESETTINGS_ENABLELOCALMODE] action create (up to date)
    * file[/opt/gitlab/etc/mattermost/env/MM_PLUGINSETTINGS_ENABLEUPLOADS] action create (up to date)
     (up to date)
  * service[mattermost] action nothing (skipped due to action :nothing)
  * runit_service[mattermost] action enable
    * ruby_block[restart_service] action nothing (skipped due to action :nothing)
    * ruby_block[restart_log_service] action nothing (skipped due to action :nothing)
    * ruby_block[reload_log_service] action nothing (skipped due to action :nothing)
    * directory[/opt/gitlab/sv/mattermost] action create (up to date)
    * template[/opt/gitlab/sv/mattermost/run] action create (up to date)
    * directory[/opt/gitlab/sv/mattermost/log] action create (up to date)
    * directory[/opt/gitlab/sv/mattermost/log/main] action create (up to date)
    * template[/opt/gitlab/sv/mattermost/log/config] action create (up to date)
    * ruby_block[verify_chown_persisted_on_mattermost] action nothing (skipped due to action :nothing)
    * link[/var/log/gitlab/mattermost/config] action create (up to date)
    * template[/opt/gitlab/sv/mattermost/log/run] action create (up to date)
    * directory[/opt/gitlab/sv/mattermost/env] action create (up to date)
    * ruby_block[Delete unmanaged env files for mattermost service] action run (skipped due to only_if)
    * template[/opt/gitlab/sv/mattermost/check] action create (skipped due to only_if)
    * template[/opt/gitlab/sv/mattermost/finish] action create (skipped due to only_if)
    * directory[/opt/gitlab/sv/mattermost/control] action create (up to date)
    * link[/opt/gitlab/init/mattermost] action create (up to date)
    * file[/opt/gitlab/sv/mattermost/down] action nothing (skipped due to action :nothing)
    * directory[/opt/gitlab/service] action create (up to date)
    * link[/opt/gitlab/service/mattermost] action create (up to date)
    * ruby_block[wait for mattermost service socket] action run (skipped due to not_if)
     (up to date)
  * version_file[Create version file for Mattermost] action create
    * file[/var/opt/gitlab/mattermost/VERSION] action create (up to date)
     (up to date)

Details of package version

Provide the package version installation details

ii  gitlab-ee      15.9.0-ee.0  amd64        GitLab Enterprise Edition (including NGINX, Postgres, Redis)

Environment details

Operating System: Debian GNU/Linux 11 (bullseye)
Installation Target, remove incorrect values:
- VM: Hyper-V
Installation Type, remove incorrect values:
- Upgrade from version long ago
Is there any other software running on the machine: no
Is this a single or multiple node installation?
Resources
- CPU: 16
- Memory total: 32GiB

Configuration details

Provide the relevant sections of /etc/gitlab/gitlab.rb

mattermost_external_url 'https://mattermost.gitlab....'
mattermost['env'] = {
    'MM_NOTIFICATIONLOGSETTINGS_FILELOCATION' => '/var/log/gitlab/mattermost',
    'MM_SERVICESETTINGS_ENABLELOCALMODE' => true,
    'MM_PLUGINSETTINGS_ENABLEUPLOADS' => true,
}
mattermost['service_address'] = "0.0.0.0"
mattermost['service_site_url'] = 'https://mattermost.gitlab....'
mattermost['file_directory'] = "/fsapp/gitlab_data/mattermost_data"
mattermost_nginx['listen_addresses'] = ['*', '[::]']
mattermost_nginx['redirect_http_to_https'] = true
mattermost_nginx['ssl_certificate'] = "/etc/ssl/certs/....pem"
mattermost_nginx['ssl_certificate_key'] = "/etc/ssl/private/....pem"
mattermost_nginx['real_ip_trusted_addresses'] = [ "..." ]
mattermost_nginx['proxy_custom_buffer_size'] = "16k"