Restore fails on artifacts and pages
Summary
Customer is unable to restore their backup without errors.
It looks like it's failing here because the $progress
variable hasn't been set to a string.
This variable is set in the following lines: https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/lib/tasks/gitlab/backup.rake#L229-237
Looking at the above code it appears that $progress
is set to an empty StringIO object when using cron, or to the stdout of the commands when run from the command line. In this scenario the customer is not using cron.
Noting that if the stdout is empty then this will likely fail. I thought this could be a permissions issue but was not able to identify any discrepancies between the customers permissions and my local installation.
Note also that when we view the contents of the backup tarball it does store artifacts and pages:
-rw------- git/git 219954204 2017-11-16 11:51 artifacts.tar.gz
-rw------- git/git 206 2017-11-16 11:51 pages.tar.gz
Steps to reproduce
We have not been able to reproduce this issue locally. I suspect this is an environmental issue where $progress
is not being set to the stdout. Is there a better way to handle this?
What is the current bug behavior?
Restore fails on artifacts and pages
What is the expected correct behavior?
Relevant logs and/or screenshots
Full stacktrace:
[root@example backups]# gitlab-rake gitlab:backup:artifacts:restore BACKUP=1510851119_2017_11_16_9.5.9-ee
rake aborted!
NoMethodError: private method puts' called for nil:NilClass
Did you mean? putc
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:158:inblock (4 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in load'
/opt/gitlab/embedded/bin/bundle:22:in<main>'
Tasks: TOP => gitlab:backup:artifacts:restore
(See full trace by running task with --trace)
[root@example backups]# gitlab-rake gitlab:backup:artifacts:restore BACKUP=1510851119_2017_11_16_9.5.9-ee --trace
** Invoke gitlab:backup:artifacts:restore (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute gitlab:backup:artifacts:restore
rake aborted!
NoMethodError: private method puts' called for nil:NilClass
Did you mean? putc
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:158:inblock (4 levels) in <top (required)>'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in block in execute'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:ineach'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:250:in execute'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:194:inblock in invoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/2.3.0/monitor.rb:214:in mon_synchronize'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:187:ininvoke_with_call_chain'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/task.rb:180:in invoke'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:152:ininvoke_task'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:108:in block (2 levels) in top_level'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:108:ineach'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:108:in block in top_level'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:117:inrun_with_threads'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:102:in top_level'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:80:inblock in run'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:178:in standard_exception_handling'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/lib/rake/application.rb:77:inrun'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/rake-12.0.0/exe/rake:27:in <top (required)>'
/opt/gitlab/embedded/bin/rake:22:inload'
/opt/gitlab/embedded/bin/rake:22:in <top (required)>'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:inload'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:74:in kernel_load'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli/exec.rb:27:inrun'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:332:in exec'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/command.rb:27:inrun'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in invoke_command'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor.rb:359:indispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:20:in dispatch'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/vendor/thor/lib/thor/base.rb:440:instart'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/cli.rb:11:in start'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/exe/bundle:34:inblock in <top (required)>'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/lib/bundler/friendly_errors.rb100in with_friendly_errors'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/bundler-1.13.7/exe/bundle:26:in<top (required)>'
/opt/gitlab/embedded/bin/bundle:22:in load'
/opt/gitlab/embedded/bin/bundle:22:in<main>'
Tasks: TOP => gitlab:backup:artifacts:restore
Output of checks
Results of GitLab environment info
Please see ZD issue.
Results of GitLab application Check
Please see ZD issue.
Possible fixes
Test for empty value in https://gitlab.com/gitlab-org/gitlab-ce/blob/9-5-stable/lib/tasks/gitlab/backup.rake#L229-237 before passing $progress
var.