Creating a project from a template doesn't work
Creating a new project from any template GitLab provides is broken as the current implementation requires shared disk access.
Steps to reproduce:
- Go to the new project form
- Click 'Create from template'
- Choose any of them and fill in other details
- Check the logs for the failure in sidekiq:
It does create the namespace:
gitaly_1 | time="2018-04-09T11:04:00Z" level=info msg="finished unary call" grpc.code=OK grpc.meta.call_site=namespace_exists grpc.meta.client_name=gitlab-web grpc.method=NamespaceExists grpc.request.Name=root/this-will-fail.git grpc.request.StorageName=default grpc.service=gitaly.NamespaceService grpc.time_ms=16 peer.address="172.19.0.5:40008" span.kind=server system=grpc
gitaly_1 | time="2018-04-09T11:04:00Z" level=info msg="finished unary call" grpc.code=OK grpc.meta.call_site=add_namespace grpc.meta.client_name=gitlab-web grpc.method=AddNamespace grpc.request.Name=root grpc.request.StorageName=default grpc.service=gitaly.NamespaceService grpc.time_ms=1 peer.address="172.19.0.5:40008" span.kind=server system=grpc
gitaly_1 | time="2018-04-09T11:04:00Z" level=info msg="finished unary call" grpc.code=OK grpc.meta.call_site=add_namespace grpc.meta.client_name=gitlab-web grpc.method=AddNamespace grpc.request.Name=root grpc.request.StorageName=default grpc.service=gitaly.NamespaceService grpc.time_ms=0 peer.address="172.19.0.5:40008" span.kind=server system=grpc
Waiting a long time. While the import is in progress
Now we'll get:
sidekiq_1 | 2018-04-09T11:08:17.589Z 13 TID-gryzar3sk RepositoryImportWorker JID-3cdad1c9c7ea4a98a0bb6b4c INFO: fail: 256.085 sec
sidekiq_1 | 2018-04-09T11:08:17.589Z 13 TID-gryzar3sk WARN: {"context":"Job raised exception","job":{"retry":false,"args":[1],"class":"RepositoryImportWorker","queue":"repository_import","backtrace":5,"status_expiration":54000,"jid":"3cdad1c9c7ea4a98a0bb6b4c","created_at":1523271841.5015013,"enqueued_at":1523271841.5034811},"jobstr":"{\"retry\":false,\"args\":[1],\"class\":\"RepositoryImportWorker\",\"queue\":\"repository_import\",\"backtrace\":5,\"status_expiration\":54000,\"jid\":\"3cdad1c9c7ea4a98a0bb6b4c\",\"created_at\":1523271841.5015013,\"enqueued_at\":1523271841.5034811}"}
sidekiq_1 | 2018-04-09T11:08:17.589Z 13 TID-gryzar3sk WARN: RuntimeError: Error importing repository into root/this-will-fail - tar: can't open '/home/git/gitlab/shared/tmp/project_exports/uploads/80b2a8da8e36ba36ce35a193442d3545': No such file or directory
sidekiq_1 | , Unable to decompress /home/git/gitlab/shared/tmp/project_exports/uploads/80b2a8da8e36ba36ce35a193442d3545 into /home/git/gitlab/shared/tmp/project_exports/root/this-will-fail/8f3581af1a9d566371b09b5f3a8b36be
sidekiq_1 | 2018-04-09T11:08:17.590Z 13 TID-gryzar3sk WARN: /home/git/gitlab/app/workers/repository_import_worker.rb:26:in `perform'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:188:in `execute_job'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
sidekiq_1 | /home/git/gitlab/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:6:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/lib/gitlab/sidekiq_middleware/shutdown.rb:52:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/server/active_record.rb:15:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sentry-raven-2.7.2/lib/raven/integrations/sidekiq.rb:9:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:133:in `invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:169:in `block in process'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:141:in `block (6 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_retry.rb:97:in `local'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:140:in `block (5 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq.rb:36:in `block in <module:Sidekiq>'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:136:in `block (4 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:204:in `stats'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:131:in `block (3 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_logger.rb:7:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:130:in `block (2 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_retry.rb:72:in `global'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:129:in `block in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/logging.rb:44:in `with_context'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/logging.rb:38:in `with_job_hash_context'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:128:in `dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:168:in `process'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:85:in `process_one'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:73:in `run'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/util.rb:16:in `watchdog'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/util.rb:25:in `block in safe_thread'
sidekiq_1 | Import/Export error raised on /home/git/gitlab/lib/gitlab/import_export/command_line_util.rb:31:in `execute': tar: can't open '/home/git/gitlab/shared/tmp/project_exports/uploads/80b2a8da8e36ba36ce35a193442d3545': No such file or directory
sidekiq_1 |
sidekiq_1 | No backtrace found
sidekiq_1 | Import/Export error raised on /home/git/gitlab/lib/gitlab/import_export/file_importer.rb:26:in `rescue in import': Unable to decompress /home/git/gitlab/shared/tmp/project_exports/uploads/80b2a8da8e36ba36ce35a193442d3545 into /home/git/gitlab/shared/tmp/project_exports/root/this-will-fail/8f3581af1a9d566371b09b5f3a8b36be
sidekiq_1 | Import/Export backtrace: /home/git/gitlab/lib/gitlab/import_export/file_importer.rb:48:in `decompress_archive'
sidekiq_1 | /home/git/gitlab/lib/gitlab/import_export/file_importer.rb:23:in `block in import'
sidekiq_1 | /home/git/gitlab/lib/gitlab/import_export/file_importer.rb:42:in `wait_for_archived_file'
sidekiq_1 | /home/git/gitlab/lib/gitlab/import_export/file_importer.rb:22:in `import'
sidekiq_1 | /home/git/gitlab/lib/gitlab/import_export/file_importer.rb:9:in `import'
sidekiq_1 | /home/git/gitlab/lib/gitlab/import_export/importer.rb:38:in `import_file'
sidekiq_1 | /home/git/gitlab/lib/gitlab/import_export/importer.rb:19:in `execute'
sidekiq_1 | /home/git/gitlab/app/services/projects/import_service.rb:81:in `import_data'
sidekiq_1 | /home/git/gitlab/app/services/projects/import_service.rb:20:in `execute'
sidekiq_1 | /home/git/gitlab/app/workers/repository_import_worker.rb:17:in `perform'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:188:in `execute_job'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
sidekiq_1 | /home/git/gitlab/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:6:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/lib/gitlab/sidekiq_middleware/shutdown.rb:52:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/server/active_record.rb:15:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sentry-raven-2.7.2/lib/raven/integrations/sidekiq.rb:9:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:133:in `invoke'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:169:in `block in process'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:141:in `block (6 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_retry.rb:97:in `local'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:140:in `block (5 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq.rb:36:in `block in <module:Sidekiq>'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:136:in `block (4 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:204:in `stats'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:131:in `block (3 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_logger.rb:7:in `call'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:130:in `block (2 levels) in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/job_retry.rb:72:in `global'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:129:in `block in dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/logging.rb:44:in `with_context'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/logging.rb:38:in `with_job_hash_context'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:128:in `dispatch'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:168:in `process'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:85:in `process_one'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:73:in `run'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/util.rb:16:in `watchdog'
sidekiq_1 | /home/git/gitlab/vendor/bundle/ruby/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/util.rb:25:in `block in safe_thread'
omnibus_1 | 172.19.0.1 - - [09/Apr/2018:11:08:20 +0000] "GET /root/this-will-fail/import HTTP/1.1" 302 118 "http://localhost:3000/root/this-will-fail/import" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:59.0) Gecko/20100101 Firefox/59.0"
Sidekiq and Unicorn are using the disk for all project imports from an archive, and from templates as they use archives.
/cc @Ahmadposten
Edited by Zeger-Jan van de Weg