"Failed to authorize your Git request: internal API unreachable" on every Git operation in Docker GitLab CE 11.9.8 setup with jwilder nginx reverse proxy
Summary
Hello,
All Git operations that I send to my repositories fail with the following error:
Fetch Failed GitLab: Failed to authorize your Git request: internal API unreachable Could not read from remote repository.
This does suggest my call is reaching GitLab, but somehow it fails to authorize my operation.
I have verified that my key is in the GitLab instance and correctly validated in the authorized_keys file.
Navigating the GitLab webpages is working fine. The certificates and such all work fine.
Background info
I have recently Dockerized my GitLab CE hosted instance. Before, it was running standalone on a VPS.
I had exported a backup of it and started setting up a Docker version through Docker Compose.
After some initial effort getting this to work withjwilder's reverse nginx proxy as well as the LetsEncrypt companion container, I was able to get GitLab to run including the certificates being handled separately.
https://github.com/jwilder/nginx-proxy https://github.com/JrCs/docker-letsencrypt-nginx-proxy-companion
Initially I had set up the same version as my backup was for. I then imported the backup, and upgraded the GitLab instance through specifying the latest Docker image in the compose file.
Perhaps important:
- This VPS does not have a large amount of resources, but GitLab worked fine before in a standalone setup.
- After the version upgrade, I had problems with the migrations not being executed. I fixed this by running them manually with gitlab-rake.
- After the backup import, I had problems with encryption secrets needing to be cleared and recreated, which I did manually through dbconsole.
Steps to reproduce
(Including the reverse proxy setup, despite suspicion that it is unrelated).
- Have jwilder's nginx-proxy running
- Have the LetsEncrypt nginx proxy companion running (network called nginx-proxy)
- Set up the following docker-compose.yml (use your own hosts and such)
Expand for docker-compose.yml contents
version: '3' services: app: image: gitlab/gitlab-ce:latest restart: always hostname: git.datavenia.nl environment: VIRTUAL_HOST: git.datavenia.nl,registry.datavenia.nl LETSENCRYPT_HOST: git.datavenia.nl,registry.datavenia.nl LETSENCRYPT_EMAIL: silvan@datavenia.nl GITLAB_OMNIBUS_CONFIG: | # The external URL's have the HTTPS scheme external_url 'https://git.datavenia.nl' registry_external_url 'https://registry.datavenia.nl' # But we let nginx-proxy handle the SSL to the outside, internally proxy talks with GitLab over HTTP letsencrypt["enable"] = false nginx["listen_port"] = 80 nginx["listen_https"] = false registry_nginx["listen_port"] = 80 registry_nginx["listen_https"] = false nginx["proxy_set_headers"] = { "X-Forwarded-Proto" => "https", "X-Forwarded-Ssl" => "on" } # Add any other gitlab.rb configuration here, each on its own line volumes: - /docker/gitlab/data/etc:/etc/gitlab - /docker/gitlab/data/var/log:/var/log/gitlab - /docker/gitlab/data/var/opt:/var/opt/gitlab networks: - entry expose: - 80 - 443 - 22 networks: # For hooking up to a reverse proxy: entry: external: name: nginx-proxy
What is the current bug behavior?
Every Git operation is rejected with an Internal API unreachable error.
What is the expected correct behavior?
The Git operation is allowed and executed.
Relevant logs and/or screenshots
Nothing is showing up in the logs when performing the Git operation. The gitlab-shell log is empty, which I find strange because the error suggests it comes from Gitlab. I don't see anything in the other logs either.
Below is the gitlab-ctl reconfigure log:
Expand for output of the gitlab-ctl reconfigure command
root@git:/# gitlab-ctl reconfigure Starting Chef Client, version 13.6.4 resolving cookbooks for run list: ["gitlab"] Synchronizing Cookbooks: - gitlab (0.0.1) - package (0.1.0) - postgresql (0.1.0) - registry (0.1.0) - redis (0.1.0) - mattermost (0.1.0) - consul (0.1.0) - gitaly (0.1.0) - letsencrypt (0.1.0) - nginx (0.1.0) - runit (4.3.0) - acme (3.1.0) - crond (0.1.0) - compat_resource (12.19.1) Installing Cookbook Gems: Compiling Cookbooks... Recipe: gitlab::default * directory[/etc/gitlab] action create (up to date) Converging 234 resources * directory[/etc/gitlab] action create (up to date) * directory[Create /var/opt/gitlab] action create (up to date) * directory[/opt/gitlab/embedded/etc] action create (up to date) * template[/opt/gitlab/embedded/etc/gitconfig] action create (up to date) Recipe: gitlab::web-server * account[Webserver user and group] action create (up to date) Recipe: gitlab::users * directory[/var/opt/gitlab] action create (up to date) * account[GitLab user and group] action create (up to date) * template[/var/opt/gitlab/.gitconfig] action create (up to date) * directory[/var/opt/gitlab/.bundle] action create (up to date) Recipe: gitlab::gitlab-shell * storage_directory[/var/opt/gitlab/.ssh] action create * ruby_block[directory resource: /var/opt/gitlab/.ssh] action run (skipped due to not_if) (up to date) * directory[/var/log/gitlab/gitlab-shell/] action create (up to date) * directory[/var/opt/gitlab/gitlab-shell] action create (up to date) * templatesymlink[Create a config.yml and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-shell/config.yml] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-shell/config.yml to /var/opt/gitlab/gitlab-shell/config.yml] action create (up to date) (up to date) * link[/opt/gitlab/embedded/service/gitlab-shell/.gitlab_shell_secret] action create (up to date) * execute[/opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-keys check-permissions] action run - execute /opt/gitlab/embedded/service/gitlab-shell/bin/gitlab-keys check-permissions * bash[Set proper security context on ssh files for selinux] action run (skipped due to only_if) Recipe: gitlab::gitlab-rails * storage_directory[/var/opt/gitlab/git-data] action create * ruby_block[directory resource: /var/opt/gitlab/git-data] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/git-data/repositories] action create * ruby_block[directory resource: /var/opt/gitlab/git-data/repositories] action run (skipped due to not_if) (up to date) * directory[/var/log/gitlab] action create (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared/artifacts] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/artifacts] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared/external-diffs] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/external-diffs] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared/lfs-objects] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/lfs-objects] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared/packages] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/packages] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/uploads] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/uploads] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-ci/builds] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-ci/builds] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared/cache] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/cache] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared/tmp] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/tmp] action run (skipped due to not_if) (up to date) * storage_directory[/var/opt/gitlab/gitlab-rails/shared/pages] action create * ruby_block[directory resource: /var/opt/gitlab/gitlab-rails/shared/pages] action run (skipped due to not_if) (up to date) * directory[create /var/opt/gitlab/gitlab-rails/etc] action create (up to date) * directory[create /opt/gitlab/etc/gitlab-rails] action create (up to date) * directory[create /var/opt/gitlab/gitlab-rails/working] action create (up to date) * directory[create /var/opt/gitlab/gitlab-rails/tmp] action create (up to date) * directory[create /var/opt/gitlab/gitlab-rails/upgrade-status] action create (up to date) * directory[create /var/log/gitlab/gitlab-rails] action create (up to date) * storage_directory[/var/opt/gitlab/backups] action create * ruby_block[directory resource: /var/opt/gitlab/backups] action run (skipped due to not_if) (up to date) * directory[/var/opt/gitlab/gitlab-rails] action create (up to date) * directory[/var/opt/gitlab/gitlab-ci] action create (up to date) * file[/var/opt/gitlab/gitlab-rails/etc/gitlab-registry.key] action create (up to date) * template[/opt/gitlab/etc/gitlab-rails/gitlab-rails-rc] action create (up to date) * file[/opt/gitlab/embedded/service/gitlab-rails/.secret] action delete (up to date) * file[/var/opt/gitlab/gitlab-rails/etc/secret] action delete (up to date) * templatesymlink[Create a database.yml and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-rails/etc/database.yml] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-rails/config/database.yml to /var/opt/gitlab/gitlab-rails/etc/database.yml] action create (up to date) (up to date) * templatesymlink[Create a secrets.yml and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-rails/etc/secrets.yml] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-rails/config/secrets.yml to /var/opt/gitlab/gitlab-rails/etc/secrets.yml] action create (up to date) (up to date) * templatesymlink[Create a resque.yml and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-rails/etc/resque.yml] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-rails/config/resque.yml to /var/opt/gitlab/gitlab-rails/etc/resque.yml] action create (up to date) (up to date) * templatesymlink[Create a redis.cache.yml and create a symlink to Rails root] action create (skipped due to not_if) * templatesymlink[Create a redis.queues.yml and create a symlink to Rails root] action create (skipped due to not_if) * templatesymlink[Create a redis.shared_state.yml and create a symlink to Rails root] action create (skipped due to not_if) * templatesymlink[Create a smtp_settings.rb and create a symlink to Rails root] action delete * file[/var/opt/gitlab/gitlab-rails/etc/smtp_settings.rb] action delete (up to date) * link[/opt/gitlab/embedded/service/gitlab-rails/config/initializers/smtp_settings.rb] action delete (up to date) (up to date) * templatesymlink[Create a gitlab.yml and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-rails/etc/gitlab.yml] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml to /var/opt/gitlab/gitlab-rails/etc/gitlab.yml] action create (up to date) (up to date) * templatesymlink[Create a rack_attack.rb and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-rails/etc/rack_attack.rb] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-rails/config/initializers/rack_attack.rb to /var/opt/gitlab/gitlab-rails/etc/rack_attack.rb] action create (up to date) (up to date) * templatesymlink[Create a gitlab_workhorse_secret and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-rails/etc/gitlab_workhorse_secret] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-rails/.gitlab_workhorse_secret to /var/opt/gitlab/gitlab-rails/etc/gitlab_workhorse_secret] action create (up to date) (up to date) * templatesymlink[Create a gitlab_shell_secret and create a symlink to Rails root] action create * template[/var/opt/gitlab/gitlab-rails/etc/gitlab_shell_secret] action create (up to date) * link[Link /opt/gitlab/embedded/service/gitlab-rails/.gitlab_shell_secret to /var/opt/gitlab/gitlab-rails/etc/gitlab_shell_secret] action create (up to date) (up to date) * templatesymlink[Create a gitlab_pages_secret and create a symlink to Rails root] action create (skipped due to only_if) * link[/opt/gitlab/embedded/service/gitlab-rails/config/initializers/relative_url.rb] action delete (up to date) * file[/var/opt/gitlab/gitlab-rails/etc/relative_url.rb] action delete (up to date) * env_dir[/opt/gitlab/etc/gitlab-rails/env] action create * directory[/opt/gitlab/etc/gitlab-rails/env] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/HOME] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/RAILS_ENV] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/LD_PRELOAD] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/SIDEKIQ_MEMORY_KILLER_MAX_RSS] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/BUNDLE_GEMFILE] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/PATH] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/ICU_DATA] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/PYTHONPATH] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/EXECJS_RUNTIME] action create (up to date) * file[/opt/gitlab/etc/gitlab-rails/env/TZ] action create (up to date) (up to date) * link[/opt/gitlab/embedded/service/gitlab-rails/tmp] action create (up to date) * link[/opt/gitlab/embedded/service/gitlab-rails/public/uploads] action create (up to date) * link[/opt/gitlab/embedded/service/gitlab-rails/log] action create (up to date) * link[/var/log/gitlab/gitlab-rails/sidekiq.log] action create (skipped due to not_if) * file[/opt/gitlab/embedded/service/gitlab-rails/db/schema.rb] action create (up to date) * remote_file[/var/opt/gitlab/gitlab-rails/VERSION] action create (up to date) * remote_file[/var/opt/gitlab/gitlab-rails/REVISION] action create (up to date) * file[/var/opt/gitlab/gitlab-rails/RUBY_VERSION] action create (up to date) * execute[clear the gitlab-rails cache] action nothing (skipped due to action :nothing) * file[/var/opt/gitlab/gitlab-rails/config.ru] action delete (up to date) Recipe: gitlab::add_trusted_certs * directory[/etc/gitlab/trusted-certs] action create (up to date) * directory[/opt/gitlab/embedded/ssl/certs] action create (up to date) * file[/opt/gitlab/embedded/ssl/certs/README] action create (up to date) * ruby_block[Move existing certs and link to /opt/gitlab/embedded/ssl/certs] action run (skipped due to only_if) Recipe: gitlab::default * service[create a temporary unicorn service] action nothing (skipped due to action :nothing) * service[create a temporary puma service] action nothing (skipped due to action :nothing) * service[create a temporary sidekiq service] action nothing (skipped due to action :nothing) * service[create a temporary mailroom service] action nothing (skipped due to action :nothing) Recipe: redis::enable * account[user and group for redis] action create (up to date) * group[Socket group] action create (up to date) * directory[/var/opt/gitlab/redis] action create (up to date) * directory[/var/log/gitlab/redis] action create (up to date) * template[/var/opt/gitlab/redis/redis.conf] action create (up to date) Recipe: * service[redis] action nothing (skipped due to action :nothing) Recipe: redis::enable * runit_service[redis] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/redis] action create (up to date) * template[/opt/gitlab/sv/redis/run] action create (up to date) * directory[/opt/gitlab/sv/redis/log] action create (up to date) * directory[/opt/gitlab/sv/redis/log/main] action create (up to date) * template[/opt/gitlab/sv/redis/log/run] action create (up to date) * template[/var/log/gitlab/redis/config] action create (up to date) * directory[/opt/gitlab/sv/redis/env] action create (up to date) * ruby_block[Delete unmanaged env files for redis service] action run (skipped due to only_if) * template[/opt/gitlab/sv/redis/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/redis/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/redis/control] action create (up to date) * link[/opt/gitlab/init/redis] action create (up to date) * file[/opt/gitlab/sv/redis/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/redis] action create (up to date) * ruby_block[wait for redis service socket] action run (skipped due to not_if) (up to date) Recipe: gitaly::enable * directory[/var/opt/gitlab/gitaly] action create (up to date) * directory[/var/log/gitlab/gitaly] action create (up to date) * env_dir[/opt/gitlab/etc/gitaly/env] action create * directory[/opt/gitlab/etc/gitaly/env] action create (up to date) * file[/opt/gitlab/etc/gitaly/env/HOME] action create (up to date) * file[/opt/gitlab/etc/gitaly/env/PATH] action create (up to date) * file[/opt/gitlab/etc/gitaly/env/TZ] action create (up to date) * file[/opt/gitlab/etc/gitaly/env/PYTHONPATH] action create (up to date) * file[/opt/gitlab/etc/gitaly/env/ICU_DATA] action create (up to date) * file[/opt/gitlab/etc/gitaly/env/SSL_CERT_DIR] action create (up to date) (up to date) * file[/opt/gitlab/etc/gitaly/HOME] action delete (up to date) * file[/opt/gitlab/etc/gitaly/PATH] action delete (up to date) * file[/opt/gitlab/etc/gitaly/TZ] action delete (up to date) * file[/opt/gitlab/etc/gitaly/PYTHONPATH] action delete (up to date) * file[/opt/gitlab/etc/gitaly/ICU_DATA] action delete (up to date) * file[/opt/gitlab/etc/gitaly/SSL_CERT_DIR] action delete (up to date) * template[Create Gitaly config.toml] action create (up to date) Recipe: * service[gitaly] action nothing (skipped due to action :nothing) Recipe: gitaly::enable * runit_service[gitaly] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/gitaly] action create (up to date) * template[/opt/gitlab/sv/gitaly/run] action create (up to date) * directory[/opt/gitlab/sv/gitaly/log] action create (up to date) * directory[/opt/gitlab/sv/gitaly/log/main] action create (up to date) * template[/opt/gitlab/sv/gitaly/log/run] action create (up to date) * template[/var/log/gitlab/gitaly/config] action create (up to date) * directory[/opt/gitlab/sv/gitaly/env] action create (up to date) * ruby_block[Delete unmanaged env files for gitaly service] action run (skipped due to only_if) * template[/opt/gitlab/sv/gitaly/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/gitaly/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/gitaly/control] action create (up to date) * link[/opt/gitlab/init/gitaly] action create (up to date) * file[/opt/gitlab/sv/gitaly/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/gitaly] action create (up to date) * ruby_block[wait for gitaly service socket] action run (skipped due to not_if) (up to date) * file[/var/opt/gitlab/gitaly/VERSION] action create (up to date) Recipe: postgresql::user * account[Postgresql user and group] action create (up to date) Recipe: postgresql::enable * directory[/var/opt/gitlab/postgresql] action create (up to date) * directory[/var/opt/gitlab/postgresql/data] action create (up to date) * directory[/var/log/gitlab/postgresql] action create (up to date) * link[/var/opt/gitlab/postgresql/data] action create (skipped due to not_if) * file[/var/opt/gitlab/postgresql/.profile] action create (up to date) * sysctl[kernel.shmmax] action create * directory[create /etc/sysctl.d for kernel.shmmax] action create (up to date) * file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.shmmax.conf kernel.shmmax] action create (up to date) * link[/etc/sysctl.d/90-omnibus-gitlab-kernel.shmmax.conf] action create (up to date) * file[delete /etc/sysctl.d/90-postgresql.conf kernel.shmmax] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-unicorn.conf kernel.shmmax] action delete (skipped due to only_if) * file[delete /opt/gitlab/embedded/etc/90-omnibus-gitlab.conf kernel.shmmax] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-omnibus-gitlab.conf kernel.shmmax] action delete (skipped due to only_if) * execute[load sysctl conf kernel.shmmax] action nothing (skipped due to action :nothing) (up to date) * sysctl[kernel.shmall] action create * directory[create /etc/sysctl.d for kernel.shmall] action create (up to date) * file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.shmall.conf kernel.shmall] action create (up to date) * link[/etc/sysctl.d/90-omnibus-gitlab-kernel.shmall.conf] action create (up to date) * file[delete /etc/sysctl.d/90-postgresql.conf kernel.shmall] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-unicorn.conf kernel.shmall] action delete (skipped due to only_if) * file[delete /opt/gitlab/embedded/etc/90-omnibus-gitlab.conf kernel.shmall] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-omnibus-gitlab.conf kernel.shmall] action delete (skipped due to only_if) * execute[load sysctl conf kernel.shmall] action nothing (skipped due to action :nothing) (up to date) * sysctl[kernel.sem] action create * directory[create /etc/sysctl.d for kernel.sem] action create (up to date) * file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.sem.conf kernel.sem] action create (up to date) * link[/etc/sysctl.d/90-omnibus-gitlab-kernel.sem.conf] action create (up to date) * file[delete /etc/sysctl.d/90-postgresql.conf kernel.sem] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-unicorn.conf kernel.sem] action delete (skipped due to only_if) * file[delete /opt/gitlab/embedded/etc/90-omnibus-gitlab.conf kernel.sem] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-omnibus-gitlab.conf kernel.sem] action delete (skipped due to only_if) * execute[load sysctl conf kernel.sem] action nothing (skipped due to action :nothing) (up to date) * execute[/opt/gitlab/embedded/bin/initdb -D /var/opt/gitlab/postgresql/data -E UTF8] action run (skipped due to not_if) * file[/var/opt/gitlab/postgresql/data/server.crt] action create (up to date) * file[/var/opt/gitlab/postgresql/data/server.key] action create (up to date) * template[/var/opt/gitlab/postgresql/data/postgresql.conf] action create (up to date) * template[/var/opt/gitlab/postgresql/data/runtime.conf] action create (up to date) * template[/var/opt/gitlab/postgresql/data/pg_hba.conf] action create (up to date) * template[/var/opt/gitlab/postgresql/data/pg_ident.conf] action create (up to date) Recipe: * service[postgresql] action nothing (skipped due to action :nothing) Recipe: postgresql::enable * runit_service[postgresql] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/postgresql] action create (up to date) * template[/opt/gitlab/sv/postgresql/run] action create (up to date) * directory[/opt/gitlab/sv/postgresql/log] action create (up to date) * directory[/opt/gitlab/sv/postgresql/log/main] action create (up to date) * template[/opt/gitlab/sv/postgresql/log/run] action create (up to date) * template[/var/log/gitlab/postgresql/config] action create (up to date) * directory[/opt/gitlab/sv/postgresql/env] action create (up to date) * ruby_block[Delete unmanaged env files for postgresql service] action run (skipped due to only_if) * template[/opt/gitlab/sv/postgresql/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/postgresql/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/postgresql/control] action create (up to date) * template[/opt/gitlab/sv/postgresql/control/t] action create (up to date) * link[/opt/gitlab/init/postgresql] action create (up to date) * file[/opt/gitlab/sv/postgresql/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/postgresql] action create (up to date) * ruby_block[wait for postgresql service socket] action run (skipped due to not_if) * directory[/opt/gitlab/service/postgresql/supervise] action create (up to date) * directory[/opt/gitlab/service/postgresql/log/supervise] action create (up to date) * file[/opt/gitlab/sv/postgresql/supervise/ok] action touch (skipped due to only_if) * file[/opt/gitlab/sv/postgresql/log/supervise/ok] action touch (skipped due to only_if) * file[/opt/gitlab/sv/postgresql/supervise/status] action touch (skipped due to only_if) * file[/opt/gitlab/sv/postgresql/log/supervise/status] action touch (skipped due to only_if) * file[/opt/gitlab/sv/postgresql/supervise/control] action touch (skipped due to only_if) * file[/opt/gitlab/sv/postgresql/log/supervise/control] action touch (skipped due to only_if) (up to date) Recipe: postgresql::bin * ruby_block[Link postgresql bin files to the correct version] action run (skipped due to only_if) Recipe: postgresql::enable * template[/opt/gitlab/etc/gitlab-psql-rc] action create (up to date) * postgresql_user[gitlab] action create * execute[create gitlab postgresql user] action run (skipped due to not_if) (up to date) * execute[create gitlabhq_production database] action run (skipped due to not_if) * postgresql_user[gitlab_replicator] action create * execute[create gitlab_replicator postgresql user] action run (skipped due to not_if) * execute[set options for gitlab_replicator postgresql user] action run (skipped due to not_if) (up to date) * postgresql_extension[pg_trgm] action enable * postgresql_query[enable pg_trgm extension] action run (skipped due to only_if) (up to date) * ruby_block[warn pending postgresql restart] action run (skipped due to only_if) * execute[reload postgresql] action nothing (skipped due to action :nothing) * execute[start postgresql] action nothing (skipped due to action :nothing) Recipe: gitlab::database_migrations * bash[migrate gitlab-rails database] action run (skipped due to not_if) Recipe: gitlab::logrotate_folders_and_configs * directory[/var/opt/gitlab/logrotate] action create (up to date) * directory[/var/opt/gitlab/logrotate/logrotate.d] action create (up to date) * directory[/var/log/gitlab/logrotate] action create (up to date) * template[/var/opt/gitlab/logrotate/logrotate.conf] action create (up to date) * template[/var/opt/gitlab/logrotate/logrotate.d/nginx] action create (up to date) * template[/var/opt/gitlab/logrotate/logrotate.d/unicorn] action create (up to date) * template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-rails] action create (up to date) * template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-shell] action create (up to date) * template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-workhorse] action create (up to date) * template[/var/opt/gitlab/logrotate/logrotate.d/gitlab-pages] action create (up to date) Recipe: gitlab::unicorn * directory[/var/log/gitlab/unicorn] action create (up to date) * directory[/opt/gitlab/var/unicorn] action create (up to date) * directory[/var/opt/gitlab/gitlab-rails/sockets] action create (up to date) * directory[/var/opt/gitlab/gitlab-rails/etc] action create (up to date) * template[/var/opt/gitlab/gitlab-rails/etc/unicorn.rb] action create (up to date) Recipe: * service[unicorn] action nothing (skipped due to action :nothing) Recipe: gitlab::unicorn * runit_service[unicorn] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/unicorn] action create (up to date) * template[/opt/gitlab/sv/unicorn/run] action create (up to date) * directory[/opt/gitlab/sv/unicorn/log] action create (up to date) * directory[/opt/gitlab/sv/unicorn/log/main] action create (up to date) * template[/opt/gitlab/sv/unicorn/log/run] action create (up to date) * template[/var/log/gitlab/unicorn/config] action create (up to date) * directory[/opt/gitlab/sv/unicorn/env] action create (up to date) * ruby_block[Delete unmanaged env files for unicorn service] action run (skipped due to only_if) * template[/opt/gitlab/sv/unicorn/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/unicorn/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/unicorn/control] action create (up to date) * template[/opt/gitlab/sv/unicorn/control/t] action create (up to date) * link[/opt/gitlab/init/unicorn] action create (up to date) * file[/opt/gitlab/sv/unicorn/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/unicorn] action create (up to date) * ruby_block[wait for unicorn service socket] action run (skipped due to not_if) (up to date) * sysctl[net.core.somaxconn] action create * directory[create /etc/sysctl.d for net.core.somaxconn] action create (up to date) * file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-net.core.somaxconn.conf net.core.somaxconn] action create (up to date) * link[/etc/sysctl.d/90-omnibus-gitlab-net.core.somaxconn.conf] action create (up to date) * file[delete /etc/sysctl.d/90-postgresql.conf net.core.somaxconn] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-unicorn.conf net.core.somaxconn] action delete (skipped due to only_if) * file[delete /opt/gitlab/embedded/etc/90-omnibus-gitlab.conf net.core.somaxconn] action delete (skipped due to only_if) * file[delete /etc/sysctl.d/90-omnibus-gitlab.conf net.core.somaxconn] action delete (skipped due to only_if) * execute[load sysctl conf net.core.somaxconn] action nothing (skipped due to action :nothing) (up to date) Recipe: * service[puma] action nothing (skipped due to action :nothing) Recipe: gitlab::puma_disable * runit_service[puma] action disable * ruby_block[disable puma] action run (skipped due to only_if) (up to date) Recipe: gitlab::sidekiq * directory[/var/log/gitlab/sidekiq] action create (up to date) Recipe: * service[sidekiq] action nothing (skipped due to action :nothing) Recipe: gitlab::sidekiq * runit_service[sidekiq] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/sidekiq] action create (up to date) * template[/opt/gitlab/sv/sidekiq/run] action create (up to date) * directory[/opt/gitlab/sv/sidekiq/log] action create (up to date) * directory[/opt/gitlab/sv/sidekiq/log/main] action create (up to date) * template[/opt/gitlab/sv/sidekiq/log/run] action create (up to date) * template[/var/log/gitlab/sidekiq/config] action create (up to date) * directory[/opt/gitlab/sv/sidekiq/env] action create (up to date) * ruby_block[Delete unmanaged env files for sidekiq service] action run (skipped due to only_if) * template[/opt/gitlab/sv/sidekiq/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/sidekiq/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/sidekiq/control] action create (up to date) * link[/opt/gitlab/init/sidekiq] action create (up to date) * file[/opt/gitlab/sv/sidekiq/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/sidekiq] action create (up to date) * ruby_block[wait for sidekiq service socket] action run (skipped due to not_if) (up to date) Recipe: gitlab::gitlab-workhorse * directory[/var/opt/gitlab/gitlab-workhorse] action create (up to date) * directory[/var/log/gitlab/gitlab-workhorse] action create (up to date) * directory[/opt/gitlab/etc/gitlab-workhorse] action create (up to date) * env_dir[/opt/gitlab/etc/gitlab-workhorse/env] action create * directory[/opt/gitlab/etc/gitlab-workhorse/env] action create (up to date) * file[/opt/gitlab/etc/gitlab-workhorse/env/PATH] action create (up to date) * file[/opt/gitlab/etc/gitlab-workhorse/env/HOME] action create (up to date) * file[/opt/gitlab/etc/gitlab-workhorse/env/SSL_CERT_DIR] action create (up to date) (up to date) Recipe: * service[gitlab-workhorse] action nothing (skipped due to action :nothing) Recipe: gitlab::gitlab-workhorse * runit_service[gitlab-workhorse] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/gitlab-workhorse] action create (up to date) * template[/opt/gitlab/sv/gitlab-workhorse/run] action create (up to date) * directory[/opt/gitlab/sv/gitlab-workhorse/log] action create (up to date) * directory[/opt/gitlab/sv/gitlab-workhorse/log/main] action create (up to date) * template[/opt/gitlab/sv/gitlab-workhorse/log/run] action create (up to date) * template[/var/log/gitlab/gitlab-workhorse/config] action create (up to date) * directory[/opt/gitlab/sv/gitlab-workhorse/env] action create (up to date) * ruby_block[Delete unmanaged env files for gitlab-workhorse service] action run (skipped due to only_if) * template[/opt/gitlab/sv/gitlab-workhorse/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/gitlab-workhorse/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/gitlab-workhorse/control] action create (up to date) * link[/opt/gitlab/init/gitlab-workhorse] action create (up to date) * file[/opt/gitlab/sv/gitlab-workhorse/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/gitlab-workhorse] action create (up to date) * ruby_block[wait for gitlab-workhorse service socket] action run (skipped due to not_if) (up to date) * file[/var/opt/gitlab/gitlab-workhorse/VERSION] action create (up to date) * template[/var/opt/gitlab/gitlab-workhorse/config.toml] action create (up to date) Recipe: * service[mailroom] action nothing (skipped due to action :nothing) Recipe: gitlab::mailroom_disable * runit_service[mailroom] action disable * ruby_block[disable mailroom] action run (skipped due to only_if) (up to date) Recipe: gitlab::nginx * directory[/var/opt/gitlab/nginx] action create (up to date) * directory[/var/opt/gitlab/nginx/conf] action create (up to date) * directory[/var/log/gitlab/nginx] action create (up to date) * link[/var/opt/gitlab/nginx/logs] action create (up to date) * template[/var/opt/gitlab/nginx/conf/gitlab-http.conf] action create (up to date) * template[/var/opt/gitlab/nginx/conf/gitlab-smartcard-http.conf] action delete (up to date) * template[/var/opt/gitlab/nginx/conf/gitlab-pages.conf] action delete (up to date) * template[/var/opt/gitlab/nginx/conf/gitlab-registry.conf] action create (up to date) * template[/var/opt/gitlab/nginx/conf/gitlab-mattermost-http.conf] action delete (up to date) * template[/var/opt/gitlab/nginx/conf/nginx-status.conf] action create (up to date) * template[/var/opt/gitlab/nginx/conf/nginx.conf] action create (up to date) Recipe: * service[nginx] action nothing (skipped due to action :nothing) Recipe: nginx::enable * runit_service[nginx] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/nginx] action create (up to date) * template[/opt/gitlab/sv/nginx/run] action create (up to date) * directory[/opt/gitlab/sv/nginx/log] action create (up to date) * directory[/opt/gitlab/sv/nginx/log/main] action create (up to date) * template[/opt/gitlab/sv/nginx/log/run] action create (up to date) * template[/var/log/gitlab/nginx/config] action create (up to date) * directory[/opt/gitlab/sv/nginx/env] action create (up to date) * ruby_block[Delete unmanaged env files for nginx service] action run (skipped due to only_if) * template[/opt/gitlab/sv/nginx/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/nginx/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/nginx/control] action create (up to date) * link[/opt/gitlab/init/nginx] action create (up to date) * file[/opt/gitlab/sv/nginx/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/nginx] action create (up to date) * ruby_block[wait for nginx service socket] action run (skipped due to not_if) (up to date) * execute[reload nginx] action nothing (skipped due to action :nothing) Recipe: * service[remote-syslog] action nothing (skipped due to action :nothing) Recipe: gitlab::remote-syslog_disable * runit_service[remote-syslog] action disable * ruby_block[disable remote-syslog] action run (skipped due to only_if) (up to date) Recipe: * service[logrotate] action nothing (skipped due to action :nothing) Recipe: gitlab::logrotate * runit_service[logrotate] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/logrotate] action create (up to date) * template[/opt/gitlab/sv/logrotate/run] action create (up to date) * directory[/opt/gitlab/sv/logrotate/log] action create (up to date) * directory[/opt/gitlab/sv/logrotate/log/main] action create (up to date) * template[/opt/gitlab/sv/logrotate/log/run] action create (up to date) * template[/var/log/gitlab/logrotate/config] action create (up to date) * directory[/opt/gitlab/sv/logrotate/env] action create (up to date) * ruby_block[Delete unmanaged env files for logrotate service] action run (skipped due to only_if) * template[/opt/gitlab/sv/logrotate/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/logrotate/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/logrotate/control] action create (up to date) * template[/opt/gitlab/sv/logrotate/control/t] action create (up to date) * link[/opt/gitlab/init/logrotate] action create (up to date) * file[/opt/gitlab/sv/logrotate/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/logrotate] action create (up to date) * ruby_block[wait for logrotate service socket] action run (skipped due to not_if) (up to date) Recipe: * service[gitlab-pages] action nothing (skipped due to action :nothing) Recipe: gitlab::gitlab-pages_disable * runit_service[gitlab-pages] action disable * ruby_block[disable gitlab-pages] action run (skipped due to only_if) (up to date) Recipe: * service[storage-check] action nothing (skipped due to action :nothing) Recipe: gitlab::storage-check_disable * runit_service[storage-check] action disable * ruby_block[disable storage-check] action run (skipped due to only_if) (up to date) Recipe: registry::enable * directory[create /var/opt/gitlab/registry] action create (up to date) * account[Docker registry user and group] action create (up to date) * directory[create /var/opt/gitlab/registry and set the owner] action create (up to date) * directory[create /var/log/gitlab/registry and set the owner] action create (up to date) * env_dir[/opt/gitlab/etc/registry/env] action create * directory[/opt/gitlab/etc/registry/env] action create (up to date) * file[/opt/gitlab/etc/registry/env/SSL_CERT_DIR] action create (up to date) (up to date) * directory[/var/opt/gitlab/gitlab-rails/shared/registry] action create (up to date) * file[/var/opt/gitlab/registry/gitlab-registry.crt] action create (up to date) * template[/var/opt/gitlab/registry/config.yml] action create (up to date) Recipe: * service[registry] action nothing (skipped due to action :nothing) Recipe: registry::enable * runit_service[registry] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/registry] action create (up to date) * template[/opt/gitlab/sv/registry/run] action create (up to date) * directory[/opt/gitlab/sv/registry/log] action create (up to date) * directory[/opt/gitlab/sv/registry/log/main] action create (up to date) * template[/opt/gitlab/sv/registry/log/run] action create (up to date) * template[/var/log/gitlab/registry/config] action create (up to date) * directory[/opt/gitlab/sv/registry/env] action create (up to date) * ruby_block[Delete unmanaged env files for registry service] action run (skipped due to only_if) * template[/opt/gitlab/sv/registry/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/registry/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/registry/control] action create (up to date) * link[/opt/gitlab/init/registry] action create (up to date) * file[/opt/gitlab/sv/registry/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/registry] action create (up to date) * ruby_block[wait for registry service socket] action run (skipped due to not_if) (up to date) * file[/var/opt/gitlab/registry/VERSION] action create (up to date) Recipe: * service[mattermost] action nothing (skipped due to action :nothing) Recipe: mattermost::disable * runit_service[mattermost] action disable * ruby_block[disable mattermost] action run (skipped due to only_if) (up to date) Recipe: gitlab::gitlab-healthcheck * template[/opt/gitlab/etc/gitlab-healthcheck-rc] action create (up to date) Recipe: * service[node-exporter] action nothing (skipped due to action :nothing) Recipe: gitlab::node-exporter_disable * runit_service[node-exporter] action disable * ruby_block[disable node-exporter] action run (skipped due to only_if) (up to date) Recipe: gitlab::gitlab-monitor * directory[/var/opt/gitlab/gitlab-monitor] action create (up to date) * directory[/var/log/gitlab/gitlab-monitor] action create (up to date) * template[/var/opt/gitlab/gitlab-monitor/gitlab-monitor.yml] action create (up to date) Recipe: * service[gitlab-monitor] action nothing (skipped due to action :nothing) Recipe: gitlab::gitlab-monitor * runit_service[gitlab-monitor] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/gitlab-monitor] action create (up to date) * template[/opt/gitlab/sv/gitlab-monitor/run] action create (up to date) * directory[/opt/gitlab/sv/gitlab-monitor/log] action create (up to date) * directory[/opt/gitlab/sv/gitlab-monitor/log/main] action create (up to date) * template[/opt/gitlab/sv/gitlab-monitor/log/run] action create (up to date) * template[/var/log/gitlab/gitlab-monitor/config] action create (up to date) * directory[/opt/gitlab/sv/gitlab-monitor/env] action create (up to date) * ruby_block[Delete unmanaged env files for gitlab-monitor service] action run (skipped due to only_if) * template[/opt/gitlab/sv/gitlab-monitor/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/gitlab-monitor/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/gitlab-monitor/control] action create (up to date) * link[/opt/gitlab/init/gitlab-monitor] action create (up to date) * file[/opt/gitlab/sv/gitlab-monitor/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/gitlab-monitor] action create (up to date) * ruby_block[wait for gitlab-monitor service socket] action run (skipped due to not_if) (up to date) Recipe: gitlab::redis-exporter * directory[/var/log/gitlab/redis-exporter] action create (up to date) * directory[/opt/gitlab/etc/redis-exporter/env] action create (up to date) * env_dir[/opt/gitlab/etc/redis-exporter/env] action create * directory[/opt/gitlab/etc/redis-exporter/env] action create (up to date) * file[/opt/gitlab/etc/redis-exporter/env/SSL_CERT_DIR] action create (up to date) (up to date) Recipe: * service[redis-exporter] action nothing (skipped due to action :nothing) Recipe: gitlab::redis-exporter * runit_service[redis-exporter] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/redis-exporter] action create (up to date) * template[/opt/gitlab/sv/redis-exporter/run] action create (up to date) * directory[/opt/gitlab/sv/redis-exporter/log] action create (up to date) * directory[/opt/gitlab/sv/redis-exporter/log/main] action create (up to date) * template[/opt/gitlab/sv/redis-exporter/log/run] action create (up to date) * template[/var/log/gitlab/redis-exporter/config] action create (up to date) * directory[/opt/gitlab/sv/redis-exporter/env] action create (up to date) * ruby_block[Delete unmanaged env files for redis-exporter service] action run (skipped due to only_if) * template[/opt/gitlab/sv/redis-exporter/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/redis-exporter/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/redis-exporter/control] action create (up to date) * link[/opt/gitlab/init/redis-exporter] action create (up to date) * file[/opt/gitlab/sv/redis-exporter/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/redis-exporter] action create (up to date) * ruby_block[wait for redis-exporter service socket] action run (skipped due to not_if) (up to date) Recipe: gitlab::prometheus_user * account[Prometheus user and group] action create (up to date) Recipe: gitlab::prometheus * directory[/var/opt/gitlab/prometheus] action create (up to date) * directory[/var/opt/gitlab/prometheus/rules] action create (up to date) * directory[/var/log/gitlab/prometheus] action create (up to date) * directory[/opt/gitlab/etc/prometheus/env] action create (up to date) * env_dir[/opt/gitlab/etc/prometheus/env] action create * directory[/opt/gitlab/etc/prometheus/env] action create (up to date) * file[/opt/gitlab/etc/prometheus/env/SSL_CERT_DIR] action create (up to date) (up to date) * link[Link prometheus executable to correct binary] action create (up to date) * execute[reload prometheus] action nothing (skipped due to action :nothing) * file[Prometheus config] action create (up to date) Recipe: * service[prometheus] action nothing (skipped due to action :nothing) Recipe: gitlab::prometheus * runit_service[prometheus] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/prometheus] action create (up to date) * template[/opt/gitlab/sv/prometheus/run] action create (up to date) * directory[/opt/gitlab/sv/prometheus/log] action create (up to date) * directory[/opt/gitlab/sv/prometheus/log/main] action create (up to date) * template[/opt/gitlab/sv/prometheus/log/run] action create (up to date) * template[/var/log/gitlab/prometheus/config] action create (up to date) * directory[/opt/gitlab/sv/prometheus/env] action create (up to date) * ruby_block[Delete unmanaged env files for prometheus service] action run (skipped due to only_if) * template[/opt/gitlab/sv/prometheus/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/prometheus/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/prometheus/control] action create (up to date) * link[/opt/gitlab/init/prometheus] action create (up to date) * file[/opt/gitlab/sv/prometheus/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/prometheus] action create (up to date) * ruby_block[wait for prometheus service socket] action run (skipped due to not_if) (up to date) * template[/var/opt/gitlab/prometheus/rules/gitlab.rules] action create (up to date) * template[/var/opt/gitlab/prometheus/rules/node.rules] action create (up to date) Recipe: gitlab::alertmanager * directory[/var/opt/gitlab/alertmanager] action create (up to date) * directory[/var/log/gitlab/alertmanager] action create (up to date) * directory[/opt/gitlab/etc/alertmanager/env] action create (up to date) * env_dir[/opt/gitlab/etc/alertmanager/env] action create * directory[/opt/gitlab/etc/alertmanager/env] action create (up to date) * file[/opt/gitlab/etc/alertmanager/env/SSL_CERT_DIR] action create (up to date) (up to date) * file[Alertmanager config] action create (up to date) Recipe: * service[alertmanager] action nothing (skipped due to action :nothing) Recipe: gitlab::alertmanager * runit_service[alertmanager] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/alertmanager] action create (up to date) * template[/opt/gitlab/sv/alertmanager/run] action create (up to date) * directory[/opt/gitlab/sv/alertmanager/log] action create (up to date) * directory[/opt/gitlab/sv/alertmanager/log/main] action create (up to date) * template[/opt/gitlab/sv/alertmanager/log/run] action create (up to date) * template[/var/log/gitlab/alertmanager/config] action create (up to date) * directory[/opt/gitlab/sv/alertmanager/env] action create (up to date) * ruby_block[Delete unmanaged env files for alertmanager service] action run (skipped due to only_if) * template[/opt/gitlab/sv/alertmanager/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/alertmanager/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/alertmanager/control] action create (up to date) * link[/opt/gitlab/init/alertmanager] action create (up to date) * file[/opt/gitlab/sv/alertmanager/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/alertmanager] action create (up to date) * ruby_block[wait for alertmanager service socket] action run (skipped due to not_if) (up to date) Recipe: gitlab::postgres-exporter * directory[/var/log/gitlab/postgres-exporter] action create (up to date) * directory[/var/opt/gitlab/postgres-exporter] action create (up to date) * env_dir[/opt/gitlab/etc/postgres-exporter/env] action create * directory[/opt/gitlab/etc/postgres-exporter/env] action create (up to date) * file[/opt/gitlab/etc/postgres-exporter/env/SSL_CERT_DIR] action create (up to date) * file[/opt/gitlab/etc/postgres-exporter/env/DATA_SOURCE_NAME] action create (up to date) (up to date) Recipe: * service[postgres-exporter] action nothing (skipped due to action :nothing) Recipe: gitlab::postgres-exporter * runit_service[postgres-exporter] action enable * ruby_block[restart_service] action nothing (skipped due to action :nothing) * ruby_block[restart_log_service] action nothing (skipped due to action :nothing) * ruby_block[reload_log_service] action nothing (skipped due to action :nothing) * directory[/opt/gitlab/sv/postgres-exporter] action create (up to date) * template[/opt/gitlab/sv/postgres-exporter/run] action create (up to date) * directory[/opt/gitlab/sv/postgres-exporter/log] action create (up to date) * directory[/opt/gitlab/sv/postgres-exporter/log/main] action create (up to date) * template[/opt/gitlab/sv/postgres-exporter/log/run] action create (up to date) * template[/var/log/gitlab/postgres-exporter/config] action create (up to date) * directory[/opt/gitlab/sv/postgres-exporter/env] action create (up to date) * ruby_block[Delete unmanaged env files for postgres-exporter service] action run (skipped due to only_if) * template[/opt/gitlab/sv/postgres-exporter/check] action create (skipped due to only_if) * template[/opt/gitlab/sv/postgres-exporter/finish] action create (skipped due to only_if) * directory[/opt/gitlab/sv/postgres-exporter/control] action create (up to date) * link[/opt/gitlab/init/postgres-exporter] action create (up to date) * file[/opt/gitlab/sv/postgres-exporter/down] action delete (up to date) * directory[/opt/gitlab/service] action create (up to date) * link[/opt/gitlab/service/postgres-exporter] action create (up to date) * ruby_block[wait for postgres-exporter service socket] action run (skipped due to not_if) (up to date) * template[/var/opt/gitlab/postgres-exporter/queries.yaml] action create (up to date) Recipe: * service[grafana] action nothing (skipped due to action :nothing) Recipe: gitlab::grafana_disable * runit_service[grafana] action disable * ruby_block[disable grafana] action run (skipped due to only_if) (up to date) Recipe: gitlab::deprecate-skip-auto-migrations * file[/etc/gitlab/skip-auto-reconfigure] action create (skipped due to only_if) * ruby_block[skip-auto-migrations deprecation] action run (skipped due to only_if)Running handlers: Running handlers complete Chef Client finished, 1/621 resources updated in 16 seconds gitlab Reconfigured!
Results of GitLab environment info
Expand for output related to GitLab environment info
root@git:/# gitlab-rake gitlab:env:infoSystem information System: Current User: git Using RVM: no Ruby Version: 2.5.3p105 Gem Version: 2.7.6 Bundler Version:1.16.6 Rake Version: 12.3.2 Redis Version: 3.2.12 Git Version: 2.18.1 Sidekiq Version:5.2.5 Go Version: unknown
GitLab information Version: 11.9.8 Revision: 48528bc Directory: /opt/gitlab/embedded/service/gitlab-rails DB Adapter: postgresql URL: https://git.datavenia.nl HTTP Clone URL: https://git.datavenia.nl/some-group/some-project.git SSH Clone URL: git@git.datavenia.nl:some-group/some-project.git Using LDAP: no Using Omniauth: yes Omniauth Providers:
GitLab Shell Version: 8.7.1 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Prior research had me verify that the internal API check passes, which it does (all checks pass).
Expand for output related to the GitLab application check
root@git:/# gitlab-rake gitlab:check SANITIZE=true Checking GitLab subtasks ...Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 8.7.1 ? ... OK (8.7.1) Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... skipped (omnibus-gitlab has no init script) Init script up-to-date? ... skipped (omnibus-gitlab has no init script) Projects have namespace: ... 1/1 ... yes 1/2 ... yes 1/3 ... yes 1/4 ... yes 1/5 ... yes 1/6 ... yes 1/7 ... yes 1/8 ... yes 1/9 ... yes 1/10 ... yes 1/11 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.5 ? ... yes (2.5.3) Git version >= 2.18.0 ? ... yes (2.18.1) Git user has default SSH configuration? ... yes Active users: ... 3
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
Possible fixes
I have attempted a lot of different potential fixes, including parts or combinations of the following config file overrides:
Expand for attempted config options
gitlab_workhorse["auth_backend"] = 'http://127.0.0.1:8080' unicorn["socket"] = '/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket' gitlab_workhorse["auth_socket"] = '/var/opt/gitlab/gitlab-rails/sockets/gitlab.socket' unicorn["listen"] = '127.0.0.1' unicorn["port"] = 8080 unicorn["tcp_nopush"] = true gitlab_shell["git_annex_enabled"] = true
This is partly attempted because I read somewhere the problem might be Ipv6 related. Output of /etc/hosts in the Docker container:
Expand for host file
root@git:/# cat /etc/hosts 127.0.0.1 localhost ::1 localhost ip6-localhost ip6-loopback fe00::0 ip6-localnet ff00::0 ip6-mcastprefix ff02::1 ip6-allnodes ff02::2 ip6-allrouters 172.18.0.15 git.datavenia.nl git
Any help or hints that might lead me in the right direction would be very much appreciated!