Resolve common GitLab.com import/export issues
Project exports allow an entire project to be exported from a GitLab instance and then imported to the same or a different instance. This is useful for backups and transferring projects from one instance to another.
On GitLab.com we have observed a number of frequent errors when importing a ~"project export". Below is a list of the most common errors.
Proposal
Resolve or improve error handling the most common issues.
Less common issues that aren't resolved will be split into separate issues with steps to reproduce for future scheduling.
Research
Sorted list of import/export issues from 27th March, 2018:
Error | Count |
---|---|
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/file_importer.rb:26:in `rescue in import': Unable to decompress /var/opt/g... | 123 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': Failed to replace protected_branches... | 80 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:31:in `execute': gzip: stdin has more than one entry-... | 59 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:31:in `execute': | 55 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/version_checker.rb:16:in `rescue in check!': Import version mismatch: Requ... | 35 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/repo_restorer.rb:18:in `rescue in restore': 13:CreateRepositoryFromBundle:... | 26 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/version_checker.rb:16:in `rescue in check!': No such file or directory @ ...VERSION | 12 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': PG::UniqueViolation: duplicate key value violates unique constraint "index_issue_assignees_on_issue_id_and_user_id... | 6 |
Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': Failed to replace priorities... | 6 |
Original investigation
This is the result of searching for `Import/Export error` on Kibana, for the last 12 hours.Error | Count |
---|---|
Merge access levels group must be blank Push access levels group must be blank (#38526 (closed)) | 6 |
Import version mismatch | 4 |
destination path '/path' already exists and is not an empty directory | 4 |
Unable to decompress | 2 |
duplicate key value violates unique constraint index_issue_assignees_on_issue_id_and_user_id
|
1 |
gzip: stdin has more than one entry--rest ignored | 1 |
duplicate key value violates unique constraint index_issues_on_project_id_and_iid
|
1 |
As we are on an old Kibana version, we can't export the errors easily, so I had to get the JSON and run some Ruby magic, which I leave here in case we want to do this in the future:
data = JSON.parse(File.read('errors.json'))
errors = data['responses'].first['hits']['hits'].map { |h| h['_source']['message'] }
parsed_errors = errors.map { |s| s.gsub(/\/var.*\.git/, '/path') }
parsed_errors.inject(Hash.new(0)) {|h, v| h[v] += 1; h}
For the last 12 hours:
{
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': Failed to replace protected_branches because one or more of the new records could not be saved. Merge access levels group must be blank Push access levels group must be blank": 6,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:40:in `execute': fatal: destination path '/path' already exists and is not an empty directory.": 4,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/file_importer.rb:26:in `rescue in import': Unable to decompress /path into /path": 1,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:40:in `execute': ": 1,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issue_assignees_on_issue_id_and_user_id\"": 1,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': Validation failed: Title already exists at group level for ... Please choose another one.": 1,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/version_checker.rb:16:in `rescue in check!': Import version mismatch: Required 0.2.2 but was 0.1.8": 1,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/version_checker.rb:16:in `rescue in check!': Import version mismatch: Required 0.2.2 but was 0.1.4": 3,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/command_line_util.rb:40:in `execute': gzip: stdin has more than one entry--rest ignored": 1,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/file_importer.rb:26:in `rescue in import': Unable to decompress /path into /path": 1,
"Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': PG::UniqueViolation: ERROR: duplicate key value violates unique constraint \"index_issues_on_project_id_and_iid\"": 1
}