Import from GitLab.com fails (destination already exists and is not an empty directory error).
Summary
I'm trying to import an export from a GitLab.com project into my EES new instance. The import fails badly and is canceled after a few sidekiq retries.
Steps to reproduce
- Export a project from Gitlab.com
- Create a new project using the import feature and the downloaded exported package.
What is the current bug behavior?
The repository import worker raises an error about a non-empty repository destination path. I tried to remove the project and ensure the destination directory is not existing before trying again, same error.
Relevant logs
Looks like there is at least two issues showing in the production.log
First one:
Sidekiq:
Error importing repository into my_group/my_project - PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_issue_assignees_on_issue_id_and_user_id" DETAIL: Key (issue_id, user_id)=(240, 3) already exists. : INSERT INTO "issue_assignees" ("user_id", "issue_id") VALUES (3, 240)
/opt/gitlab/embedded/service/gitlab-rails/app/workers/repository_import_worker.rb:23:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:188:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'
production.log
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': PG::UniqueViolation: ERROR: duplicate key value violates unique constraint "index_issue_assignees_on_issue_id_and_user_id"
DETAIL: Key (issue_id, user_id)=(240, 3) already exists.
: INSERT INTO "issue_assignees" ("user_id", "issue_id") VALUES (3, 240)
...
/opt/gitlab/embedded/service/gitlab-rails/config/initializers/active_record_locking.rb:12:in `_create_record'
...
/opt/gitlab/embedded/service/gitlab-rails/app/models/project.rb:1614:in `append_or_update_attribute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:71:in `save_relation_hash'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:123:in `block in create_sub_relations'
...
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:116:in `create_sub_relations'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:57:in `block in create_relations'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:55:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:55:in `create_relations'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:29:in `block (2 levels) in restore'
...
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:28:in `block in restore'
...
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:27:in `restore'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/importer.rb:16:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/importer.rb:16:in `all?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/importer.rb:16:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/services/projects/import_service.rb:77:in `import_data'
/opt/gitlab/embedded/service/gitlab-rails/app/services/projects/import_service.rb:20:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/repository_import_worker.rb:17:in `perform'
...
Second error :
Sidekiq:
Error importing repository into my_group/my_project - fatal: destination path '/var/opt/gitlab/git-data/repositories/my_group/my_project.git' already exists and is not an empty directory. , undefined method `join' for nil:NilClass Did you mean? JSON
/opt/gitlab/embedded/service/gitlab-rails/app/workers/repository_import_worker.rb:23:in `perform'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:188:in `execute_job'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'
/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.5/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'
production.log
mport/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:40:in `execute': fatal: destination path '/var/opt/gitlab/git-data/repositories/my_group/my_project.git' already exists and is not an empty directory.
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/repo_restorer.rb:18:in `rescue in restore': undefined method `join' for nil:NilClass
Did you mean? JSON
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/shared.rb:23:in `error'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:40:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:19:in `git_clone_bundle'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/repo_restorer.rb:16:in `restore'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/importer.rb:16:in `each'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/importer.rb:16:in `all?'
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/importer.rb:16:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/services/projects/import_service.rb:77:in `import_data'
/opt/gitlab/embedded/service/gitlab-rails/app/services/projects/import_service.rb:20:in `execute'
/opt/gitlab/embedded/service/gitlab-rails/app/workers/repository_import_worker.rb:17:in `perform'
...
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'
...
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:6:in `call'
...
/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:16:in `call'
...
Output of checks
Results of GitLab environment info
System information
System: Ubuntu 16.04
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.3.6p384
Gem Version: 2.6.13
Bundler Version:1.13.7
Rake Version: 12.3.0
Redis Version: 3.2.11
Git Version: 2.14.3
Sidekiq Version:5.0.5
Go Version: unknown
GitLab information
Version: 10.4.0-ee
Revision: 00592f0
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: postgresql
DB Version: 9.6.5
URL: [...]
HTTP Clone URL: [...]
SSH Clone URL: [...]
Elasticsearch: yes
Geo: no
Using LDAP: no
Using Omniauth: no
GitLab Shell
Version: 5.11.0
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks
Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
All gitlab-rake gitlab:check
tests passed.
Edited by Simon Elcham