Import of "Gitlab export" fails because release description can't be blank
Summary
Importing a previously exported GitLab Export fails with the error:
Failed to replace releases because one or more of the new records could not be saved. Description can't be blank
Steps to reproduce
- Export a project
- Create a new project with importing from a
GitLab export
Example Project
I've imported a private GitHub export with a patched version of my omnibus CE 10.0.3 8895150. Hence, the root-cause might be that the description on releases should never have been empty in the first place.
What is the current bug behavior?
Import fails entirely.
What is the expected correct behavior?
Import should proceed. Either ignoring the validation or e.g. replace empty descriptions with the tag property (which would be more difficult to implement, as the json tree is parsed without the knowledge of it's content).
Relevant logs and/or screenshots
project.json
... that would make the import fail (see the release with the empty_description
tag and empty description):
{
"description": "Example Project",
...
"releases": [{
"id": 1,
"tag": "OK",
"description": "As long as this discription is not empty, all is good.",
"project_id": 1,
"created_at": "2017-09-25T10:26:04.000Z",
"updated_at": "2017-10-14T19:26:54.296Z"
},
{
"id": 2,
"tag": "empty_description",
"description": "",
"project_id": 1,
"created_at": "2017-09-14T10:03:11.000Z",
"updated_at": "2017-10-14T19:26:54.311Z"
},
...
}
Logs
/var/log/gitlab/gitlab-rails/production.log
``` Import/Export error raised on /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:33:in `rescue in restore': Failed to replace releases because one or more of the new records could not be saved. Description can't be blank /opt/gitlab/embedded/service/gitlab-rails.patch/app/models/project.rb:1668:in `handle_update_attribute_error' /opt/gitlab/embedded/service/gitlab-rails.patch/app/models/project.rb:1480:in `rescue in append_or_update_attribute' /opt/gitlab/embedded/service/gitlab-rails.patch/app/models/project.rb:1471:in `append_or_update_attribute' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:69:in `save_relation_hash' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:59:in `block in create_relations' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:55:in `each' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:55:in `create_relations' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:29:in `block (2 levels) in restore' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/no_touching.rb:29:in `apply_to' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/no_touching.rb:22:in `no_touching' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:28:in `block in restore' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/connection_adapters/abstract/query_cache.rb:49:in `uncached' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/activerecord-4.2.8/lib/active_record/query_cache.rb:19:in `uncached' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/project_tree_restorer.rb:27:in `restore' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/importer.rb:12:in `each' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/importer.rb:12:in `all?' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/import_export/importer.rb:12:in `execute' /opt/gitlab/embedded/service/gitlab-rails.patch/app/services/projects/import_service.rb:79:in `import_data' /opt/gitlab/embedded/service/gitlab-rails.patch/app/services/projects/import_service.rb:16:in `execute' /opt/gitlab/embedded/service/gitlab-rails.patch/app/workers/repository_import_worker.rb:20:in `perform' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:199:in `execute_job' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:170:in `block (2 levels) in process' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:128:in `block in invoke' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/sidekiq_middleware/request_store_middleware.rb:6:in `call' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' /opt/gitlab/embedded/service/gitlab-rails.patch/lib/gitlab/sidekiq_middleware/memory_killer.rb:17:in `call' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/server/active_record.rb:15:in `call' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sentry-raven-2.5.3/lib/raven/integrations/sidekiq.rb:7:in `call' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:130:in `block in invoke' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:133:in `invoke' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:169:in `block in process' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:141:in `block (6 levels) in dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/job_retry.rb:97:in `local' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:140:in `block (5 levels) in dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq.rb:36:in `block in ' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:136:in `block (4 levels) in dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:215:in `stats' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:131:in `block (3 levels) in dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/job_logger.rb:7:in `call' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:130:in `block (2 levels) in dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/job_retry.rb:72:in `global' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:129:in `block in dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/logging.rb:44:in `with_context' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/logging.rb:38:in `with_job_hash_context' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:128:in `dispatch' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:168:in `process' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:85:in `process_one' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:73:in `run' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/util.rb:16:in `watchdog' /opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/util.rb:25:in `block in safe_thread'
Results of GitLab environment info
`sudo gitlab-rake gitlab:env:info`
System information System: Ubuntu 16.04 Current User: git Using RVM: no Ruby Version: 2.3.5p376 Gem Version: 2.6.13 Bundler Version:1.13.7 Rake Version: 12.0.0 Redis Version: 3.2.5 Git Version: 2.13.5 Sidekiq Version:5.0.4 Go Version: unknownGitLab information Version: 10.0.3 Revision: 8895150 Directory: /opt/gitlab/embedded/service/gitlab-rails.patch DB Adapter: postgresql URL: https://gitlab.metaworx.ch HTTP Clone URL: https://gitlab.metaworx.ch/some-group/some-project.git SSH Clone URL: [git@gitlab.metaworx.ch:5544]:some-group/some-project.git Using LDAP: no Using Omniauth: no
GitLab Shell Version: 5.9.0 Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
Expand for output related to the GitLab application check
Checking GitLab Shell ...GitLab Shell version >= 5.9.0 ? ... OK (5.9.0) Repo base directory exists? default... yes Repo storage directories are symlinks? default... no Repo paths owned by git:root, or git:git? default... yes Repo paths access is drwxrws---? default... yes hooks directories in repos are links: ... 3/7 ... ok Running /opt/gitlab/embedded/service/gitlab-shell/bin/check Check GitLab API access: OK Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Sidekiq ...
Running? ... yes Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Reply by email is disabled in config/gitlab.yml Checking LDAP ...
LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab ...
Git configured correctly? ... yes Database config exists? ... yes All migrations up? ... yes Database contains orphaned GroupMembers? ... no GitLab config exists? ... yes GitLab config up to date? ... yes Log directory writable? ... yes Tmp directory writable? ... yes Uploads directory exists? ... yes Uploads directory has correct permissions? ... yes Uploads directory tmp has correct permissions? ... yes Init script exists? ... no Try fixing it: Install the init script For more information see: doc/install/installation.md in section "Install Init Script" Please fix the error above and rerun the checks. Init script up-to-date? ... can't check because of previous errors Projects have namespace: ... 3/7 ... yes Redis version >= 2.8.0? ... yes Ruby version >= 2.3.3 ? ... yes (2.3.5) Git version >= 2.7.3 ? ... yes (2.13.5) Git user has default SSH configuration? ... no Try fixing it: mkdir ~/gitlab-check-backup-1508083202 sudo mv /var/opt/gitlab/.ssh/authorized_keys.lock ~/gitlab-check-backup-1508083202 For more information see: doc/ssh/README.md in section "SSH on the GitLab server" Please fix the error above and rerun the checks. Active users: ... 2
Checking GitLab ... Finished
Possible fixes
Maybe here? https://gitlab.com/gitlab-org/gitlab-ce/blob/master/app/models/release.rb#L8
Also refer to: Make Import/Export more resilient to errors