GitLab CE GitHub import fails for no apparent reason

Summary

When trying to import a project from GitHub into a self hosted GitLab CE instance (10.1) the import fails for no apparent reason. It feels like a regression to me because I imported already over 100 projects on a previous version of GitLab CE. Especially one of the now failing projects went through two weeks ago on an older Version of GitLab CE.

Steps to reproduce

  1. Go to create new project
  2. Choose "import from GitHub"
  3. Select project to import

Example Project

My feeling is the issue could be related to the size of the project to import. It fails for me now on two pretty big projects and it worked for smaller projects.

What is the current bug behavior?

After a while the import states "failed" and when looking at the project one can see something like this "Files (0 Bytes) Commits (0) Branches (8) Tags (99)"

What is the expected correct behavior?

The project should be correctly imported.

Relevant logs and/or screenshots

The sidekiq log shows:

2017-10-24_06:44:16.56340 2017-10-24T06:44:16.563Z 21709 TID-ov217c4jk WARN: {"context":"Job raised exception","job":{"class":"RepositoryImportWorker","args":[114],"retry":3,"queue":"repository_import","backtrace":5,"status_expiration":54000,"jid":"d08325cbead63e70d1a550fa","created_at":1508827129.7070837,"enqueued_at":1508827129.7075062,"error_message":"Error importing repository https://xxxxxxxx@github.com/origammi/yyyy.git into sandrinr/yyyy - The remote data could not be imported.","error_class":"RepositoryImportWorker::ImportError","failed_at":1508827456.5628977,"retry_count":0,"error_backtrace":["/opt/gitlab/embedded/service/gitlab-rails/app/workers/repository_import_worker.rb:20:in `perform'","/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:199:in `execute_job'","/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/processor.rb:170:in `block (2 levels) in process'","/opt/gitlab/embedded/lib/ruby/gems/2.3.0/gems/sidekiq-5.0.4/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'","/opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_status/server_middleware.rb:5:in `call'"]},"jobstr":"{\"class\":\"RepositoryImportWorker\",\"args\":[114],\"retry\":3,\"queue\":\"repository_import\",\"backtrace\":5,\"status_expiration\":54000,\"jid\":\"d08325cbead63e70d1a550fa\",\"created_at\":1508827129.7070837,\"enqueued_at\":1508827129.7075062}"}

Output of checks

When I try the URL used in the log above to fetch the source myself it works out perfectly.

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:		
Current User:	git
Using RVM:	no
Ruby Version:	2.3.5p376
Gem Version:	2.6.13
Bundler Version:1.13.7
Rake Version:	12.1.0
Redis Version:	3.2.5
Git Version:	2.13.5
Sidekiq Version:5.0.4
Go Version:	unknown

GitLab information
Version:	10.1.0
Revision:	5a695c4
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	postgresql
URL:		https://gitlab.origammi.io
HTTP Clone URL:	https://gitlab.origammi.io/some-group/some-project.git
SSH Clone URL:	ssh://git@gitlab.origammi.io:2203/some-group/some-project.git
Using LDAP:	no
Using Omniauth:	no


GitLab Shell
Version:	5.9.3
Repository storage paths:
  • default: /var/opt/gitlab/git-data/repositories Hooks: /opt/gitlab/embedded/service/gitlab-shell/hooks Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
Checking GitLab Shell ...

GitLab Shell version >= 5.9.3 ? ... OK (5.9.3)
Repo base directory exists?
default... yes
Repo storage directories are symlinks?
default... no
Repo paths owned by git:root, or git:git?
default... yes
Repo paths access is drwxrws---?
default... yes
hooks directories in repos are links: ...
3/2 ... ok
3/7 ... ok
3/8 ... ok
3/9 ... ok
3/10 ... ok
3/11 ... ok
3/12 ... ok
3/13 ... ok
3/14 ... ok
3/15 ... ok
3/16 ... ok
3/17 ... ok
3/18 ... ok
3/19 ... ok
3/20 ... ok
3/21 ... ok
3/22 ... ok
3/23 ... ok
3/24 ... ok
3/25 ... ok
3/26 ... ok
3/27 ... ok
3/28 ... ok
3/29 ... ok
3/30 ... ok
3/31 ... ok
3/32 ... ok
3/33 ... ok
3/34 ... ok
3/35 ... ok
3/36 ... ok
3/37 ... ok
3/38 ... ok
3/39 ... ok
3/40 ... ok
3/41 ... ok
3/42 ... ok
3/43 ... ok
3/44 ... ok
3/45 ... ok
3/46 ... ok
3/47 ... ok
3/48 ... ok
3/49 ... ok
3/50 ... ok
3/51 ... ok
3/52 ... ok
3/53 ... ok
3/54 ... ok
3/55 ... ok
3/56 ... ok
3/57 ... ok
3/58 ... ok
3/59 ... ok
3/60 ... ok
3/61 ... ok
3/62 ... ok
3/63 ... ok
3/64 ... ok
3/65 ... ok
3/66 ... ok
3/67 ... ok
3/68 ... ok
3/69 ... ok
3/70 ... ok
3/71 ... ok
3/72 ... ok
3/73 ... ok
3/74 ... ok
3/75 ... ok
3/76 ... ok
3/78 ... ok
3/79 ... ok
3/80 ... ok
3/81 ... ok
3/82 ... ok
3/83 ... ok
3/84 ... ok
3/85 ... ok
3/86 ... ok
3/87 ... ok
3/88 ... ok
3/90 ... ok
3/91 ... ok
3/92 ... ok
3/93 ... ok
3/94 ... ok
3/95 ... ok
3/96 ... ok
3/97 ... ok
3/98 ... ok
3/99 ... ok
3/100 ... ok
3/101 ... ok
3/102 ... ok
3/103 ... ok
3/104 ... ok
3/105 ... ok
3/106 ... ok
3/107 ... ok
2/108 ... ok
2/110 ... ok
2/114 ... ok
2/115 ... ok
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK


Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Sidekiq ...


Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Reply by email is disabled in config/gitlab.yml
Checking LDAP ...


LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
3/2 ... yes
3/7 ... yes
3/8 ... yes
3/9 ... yes
3/10 ... yes
3/11 ... yes
3/12 ... yes
3/13 ... yes
3/14 ... yes
3/15 ... yes
3/16 ... yes
3/17 ... yes
3/18 ... yes
3/19 ... yes
3/20 ... yes
3/21 ... yes
3/22 ... yes
3/23 ... yes
3/24 ... yes
3/25 ... yes
3/26 ... yes
3/27 ... yes
3/28 ... yes
3/29 ... yes
3/30 ... yes
3/31 ... yes
3/32 ... yes
3/33 ... yes
3/34 ... yes
3/35 ... yes
3/36 ... yes
3/37 ... yes
3/38 ... yes
3/39 ... yes
3/40 ... yes
3/41 ... yes
3/42 ... yes
3/43 ... yes
3/44 ... yes
3/45 ... yes
3/46 ... yes
3/47 ... yes
3/48 ... yes
3/49 ... yes
3/50 ... yes
3/51 ... yes
3/52 ... yes
3/53 ... yes
3/54 ... yes
3/55 ... yes
3/56 ... yes
3/57 ... yes
3/58 ... yes
3/59 ... yes
3/60 ... yes
3/61 ... yes
3/62 ... yes
3/63 ... yes
3/64 ... yes
3/65 ... yes
3/66 ... yes
3/67 ... yes
3/68 ... yes
3/69 ... yes
3/70 ... yes
3/71 ... yes
3/72 ... yes
3/73 ... yes
3/74 ... yes
3/75 ... yes
3/76 ... yes
3/78 ... yes
3/79 ... yes
3/80 ... yes
3/81 ... yes
3/82 ... yes
3/83 ... yes
3/84 ... yes
3/85 ... yes
3/86 ... yes
3/87 ... yes
3/88 ... yes
3/90 ... yes
3/91 ... yes
3/92 ... yes
3/93 ... yes
3/94 ... yes
3/95 ... yes
3/96 ... yes
3/97 ... yes
3/98 ... yes
3/99 ... yes
3/100 ... yes
3/101 ... yes
3/102 ... yes
3/103 ... yes
3/104 ... yes
3/105 ... yes
3/106 ... yes
3/107 ... yes
2/108 ... yes
2/110 ... yes
2/114 ... yes
2/115 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.3.3 ? ... yes (2.3.5)
Git version >= 2.7.3 ? ... yes (2.13.5)
Git user has default SSH configuration? ... yes
Active users: ... 18


Checking GitLab ... Finished
Edited by Raffaele Sandrini