Redis node fails to upgrade from 15.8.1 to 15.8.3
Summary
A customer ran into an issue when attempting to upgrade their Omnibus Redis node from 15.8.1 to 15.8.3. The Redis node doesn’t have a /var/opt/gitlab/gitlab-rails/etc directory
so they get a EnclosingDirectoryDoesNotExist
error.
Relevant logs
- yum update failure:
Downloading Packages:
gitlab-ee-15.8.3-ee.0.el8.x86\_64.rpm 74 MB/s | 1.2 GB 00:16
-------------------------------------------------------------------------------------------------------------------
Total 74 MB/s | 1.2 GB 00:16
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
Preparing : 1/1
Running scriptlet: gitlab-ee-15.8.3-ee.0.el8.x86\_64 1/2
Malformed configuration JSON file found at /opt/gitlab/embedded/nodes/hq2-git-redis00.paycomhq.com.json.
This usually happens when your last run of \`gitlab-ctl reconfigure\` didn't complete successfully.
This file is used to check if any of the unsupported configurations are enabled,
and hence require a working reconfigure before upgrading.
Please run \`sudo gitlab-ctl reconfigure\` to fix it and try again.
error: %prein(gitlab-ee-15.8.3-ee.0.el8.x86\_64) scriptlet failed, exit status 1Error in PREIN scriptlet in rpm package gitlab-ee
Verifying : gitlab-ee-15.8.3-ee.0.el8.x86\_64 1/2
Verifying : gitlab-ee-15.8.1-ee.0.el8.x86\_64 2/2
Installed products updated.Failed:
gitlab-ee-15.8.1-ee.0.el8.x86\_64 gitlab-ee-15.8.3-ee.0.el8.x86\_64Error: Transaction failed
- Ran
gitlab-ctl reconfigure
as suggested in the output gave this error:
Running handlers complete
\[2023-02-25T09:52:49-06:00\] ERROR: Exception handlers complete
Infra Phase failed. 4 resources updated in 01 minutes 18 seconds
\[2023-02-25T09:52:49-06:00\] FATAL: Stacktrace dumped to /opt/gitlab/embedded/cookbooks/cache/cinc-stacktrace.out
\[2023-02-25T09:52:49-06:00\] FATAL: ---------------------------------------------------------------------------------------
\[2023-02-25T09:52:49-06:00\] FATAL: PLEASE PROVIDE THE CONTENTS OF THE stacktrace.out FILE (above) IF YOU FILE A BUG REPORT
\[2023-02-25T09:52:49-06:00\] FATAL: ---------------------------------------------------------------------------------------
\[2023-02-25T09:52:49-06:00\] FATAL: Chef::Exceptions::EnclosingDirectoryDoesNotExist: templatesymlink\[Create a gitlab\_suggested\_reviewers\_secret and create a symlink to Rails root\] (gitlab-ee::suggested\_reviewers line 30) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: template\[/var/opt/gitlab/gitlab-rails/etc/gitlab\_suggested\_reviewers\_secret\] (gitlab-ee::suggested\_reviewers line 36) had an error: Chef::Exceptions::EnclosingDirectoryDoesNotExist: Parent directory /var/opt/gitlab/gitlab-rails/etc does not exist.
\[sai.ram.nukala@hq2-git-redis00 ~\]$ sudo cd /var/opt/gitlab/gitlab-rails/etc
/bin/cd: line 2: cd: /var/opt/gitlab/gitlab-rails/etc: No such file or directory
\[sai.ram.nukala@hq2-git-redis00 ~\]$ sudo cd /var/opt/gitlab/gitlab-rails
/bin/cd: line 2: cd: /var/opt/gitlab/gitlab-rails: No such file or directory
- stacktrace: StackTrace_ReddisServerdc2
Configuration details
Provide the relevant sections of `/etc/gitlab/gitlab.rb`
external_url '' # Listen on all addresses redis['bind'] = '0.0.0.0' # Set the defaul port, must be set. redis['port'] = # Set password, as in the secret `gitlab-redis` populated in Kubernetes redis['password'] = 'REDIS_PASSWORD' # On Deploy Replace## Configure Geo gitlab_rails['geo_node_name'] = '<GEO_NODE_NAME>'
# Monitoring node_exporter['enable'] = true node_exporter['listen_address'] = '0.0.0.0:' redis_exporter['enable'] = true redis_exporter['listen_address'] = '0.0.0.0:'
## Disable everything else gitlab_rails['enable'] = false sidekiq['enable'] = false puma['enable']=false registry['enable'] = false gitaly['enable'] = false gitlab_workhorse['enable'] = false nginx['enable'] = false prometheus_monitoring['enable'] = false postgresql['enable'] = false gitlab_kas['enable'] = false
Possible Workarounds
Create the folder /var/opt/gitlab/gitlab-rails
.