Unmigrated Data Check pre install action can hang indefinitely when Redis isn't available
Summary
When making a change to colocate Redis Sentinel with Redis we notice various environments failed to upgrade today due to a pre-install check for unmigrated legacy data with the rake tasks gitlab:storage:legacy_attachments
and gitlab:storage:legacy_projects
After some investigation this appears to be due to Redis not being available on an existing environment
This is a catch 22 problem then. If Redis has had it's connection details changed a reconfigure needs to happen before an Ominbus install. This hasn't been a strict requirement before - components needing to be available during the package install - since the install typically only installs the files that reconfigure
then actually sets up.
With the above now users need to reconfigure first, then install and then reconfigure again - which shouldn't be the case, as per our docs.
Steps to reproduce
Install the latest Omnibus package on an existing environment's Rails or Sidekiq nodes when Redis isn't available. Or run the rake tasks gitlab:storage:legacy_attachments
and gitlab:storage:legacy_projects
in the same situation.
What is the current bug behavior?
The unmigrated data check hangs indefinitely with out output to what the problem is.
What is the expected correct behavior?
Check should be handling this scenario gracefully and be skipped if Redis isn't available. This may be an issue with the rake tasks themselves.
Other potential solution is to do the check as part of reconfigure after the Redis connection details have been updated.
Relevant logs
apt-get install gitlab-ee
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be upgraded:
gitlab-ee
1 upgraded, 0 newly installed, 0 to remove and 5 not upgraded.
1 not fully installed or removed.
Need to get 0 B/1003 MB of archives.
After this operation, 4082 kB of additional disk space will be used.
(Reading database ... 236617 files and directories currently installed.)
Preparing to unpack .../gitlab-ee_14.2.0+rnightly.207655.77e3c772-0_amd64.deb ...
gitlab preinstall:
gitlab preinstall: This node does not appear to be running a database
gitlab preinstall: Skipping version check, if you think this is an error exit now
gitlab preinstall:
gitlab preinstall: Checking for unmigrated data on legacy storage