Gitlab CE keeps spinning with github importer
Hi,
I am using the latest Gitlab CE , on RHEl7.4.
When I try to import a repo from github using the github importer , it succeeds on those repos that have just files and commits.
It just hangs forever on those repos that are larger and have issues and pull requests.
The repo I am trying to import is:
https://github.com/iahmad-khan/ai-tools
Import in progress
git clone --bare https://*****@github.com/iahmad-khan/ai-tools.git
Please wait while we import the repository for you. Refresh at will.
Please see the screenshot.
Any idea what might be causing this.
In the background jobs , there is an entry in the Retry queue with error messege , related to: GRPC timeout.
I tested on changing alot of timeout parameters and RSS sizes etc ., but nothing worked.
Note: I can import the same repo from github in a few seconds using the url method and it works with no issue.But that approach does not bring the issues and pull requests info.
How to re-produce:
Please try to import the above repo using latest GitlabCe 10.8.3 , I installed it two days ago.
Why its trying to import the wiki , mayebe its stuck on that.
2018-06-07_17:02:18.38005 2018-06-07T17:02:18.379Z 7757 TID-osfsfv98c WARN: {"context":"Job raised exception","job":{"class":"Gitlab::GithubImport::Stage::ImportRepositoryWorker","args":[232],"retry":5,"queue":"github_importer:github_import_stage_import_repository","queue_namespace":"github_importer","dead":false,"jid":"544a8c41fcc0a0611ad32bf7","created_at":1528390714.5787973,"enqueued_at":1528390837.5994034,"error_message":"4:Deadline Exceeded","error_class":"GRPC::DeadlineExceeded","failed_at":1528390816.7237084,"retry_count":0},"jobstr":"{\"class\":\"Gitlab::GithubImport::Stage::ImportRepositoryWorker\",\"args\":[232],\"retry\":5,\"queue\":\"github_importer:github_import_stage_import_repository\",\"queue_namespace\":\"github_importer\",\"dead\":false,\"jid\":\"544a8c41fcc0a0611ad32bf7\",\"created_at\":1528390714.5787973,\"enqueued_at\":1528390837.5994034,\"error_message\":\"4:Deadline Exceeded\",\"error_class\":\"GRPC::DeadlineExceeded\",\"failed_at\":1528390816.7237084,\"retry_count\":0}"}
2018-06-07_17:02:18.38024 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/github_import/importer/repository_importer.rb:20:in `import_wiki?'
2018-06-07_17:02:18.38025 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/github_import/importer/repository_importer.rb:41:in `execute'
2018-06-07_17:02:18.38025 /opt/gitlab/embedded/service/gitlab-rails/app/workers/gitlab/github_import/stage/import_repository_worker.rb:22:in `import'
2018-06-07_17:02:18.38029 /opt/gitlab/embedded/service/gitlab-rails/app/workers/concerns/gitlab/github_import/stage_methods.rb:18:in `try_import'
2018-06-07_17:02:18.38035 /opt/gitlab/embedded/service/gitlab-rails/app/workers/concerns/gitlab/github_import/stage_methods.rb:12:in `perform'
2018-06-07_17:02:57.14947 2018-06-07T17:02:57.149Z 7757 TID-osfsfv6u8 Gitlab::GithubImport::RefreshImportJidWorker JID-51ebdae5c60d71397f766ee2 INFO: start
2018-06-07_17:02:57.16557 2018-06-07T17:02:57.165Z 7757 TID-osfsfv6u8 Gitlab::GithubImport::RefreshImportJidWorker JID-51ebdae5c60d71397f766ee2 INFO: done: 0.016 sec
2018-06-07_17:03:14.25338 2018-06-07T17:03:14.253Z 7757 TID-osfmob8oo Gitlab::GithubImport::Stage::ImportRepositoryWorker JID-544a8c41fcc0a0611ad32bf7 INFO: start
2018-06-07_17:03:14.25563 2018-06-07T17:03:14.255Z 7757 TID-osfsfvuqs Gitlab::GithubImport::RefreshImportJidWorker JID-576b690d54b0ced69d93164a INFO: start
2018-06-07_17:03:14.28574 2018-06-07T17:03:14.285Z 7757 TID-osfsfvuqs Gitlab::GithubImport::RefreshImportJidWorker JID-576b690d54b0ced69d93164a INFO: done: 0.03 sec
2018-06-07_17:03:58.38370 2018-06-07T17:03:58.382Z 7757 TID-osfsfvay0 Gitlab::GithubImport::RefreshImportJidWorker JID-b9e842e14838d90c7bd6097d INFO: start
2018-06-07_17:03:58.39885 2018-06-07T17:03:58.398Z 7757 TID-osfsfvay0 Gitlab::GithubImport::RefreshImportJidWorker JID-b9e842e14838d90c7bd6097d INFO: done: 0.016 sec
2018-06-07_17:04:25.75165 2018-06-07T17:04:25.751Z 7757 TID-osfmobeo8 Gitlab::GithubImport::RefreshImportJidWorker JID-302b3c3fcef52ab1d0419680 INFO: start
2018-06-07_17:04:25.75274 2018-06-07T17:04:25.751Z 7757 TID-osfsfvuqs Gitlab::GithubImport::RefreshImportJidWorker JID-5ecbb4ab3ba49a63f21857f2 INFO: start
2018-06-07_17:04:25.76778 2018-06-07T17:04:25.767Z 7757 TID-osfmobeo8 Gitlab::GithubImport::RefreshImportJidWorker JID-302b3c3fcef52ab1d0419680 INFO: done: 0.016 sec
2018-06-07_17:04:25.76845 2018-06-07T17:04:25.768Z 7757 TID-osfsfvuqs Gitlab::GithubImport::RefreshImportJidWorker JID-5ecbb4ab3ba49a63f21857f2 INFO: done: 0.017 sec
2018-06-07_17:04:55.13294 2018-06-07T17:04:55.132Z 7757 TID-osfmob8oo Gitlab::GithubImport::Stage::ImportRepositoryWorker JID-544a8c41fcc0a0611ad32bf7 INFO: fail: 100.88 sec
2018-06-07_17:04:55.13361 2018-06-07T17:04:55.132Z 7757 TID-osfmob8oo WARN: {"context":"Job raised exception","job":{"class":"Gitlab::GithubImport::Stage::ImportRepositoryWorker","args":[232],"retry":5,"queue":"github_importer:github_import_stage_import_repository","queue_namespace":"github_importer","dead":false,"jid":"544a8c41fcc0a0611ad32bf7","created_at":1528390714.5787973,"enqueued_at":1528390994.2521996,"error_message":"4:Deadline Exceeded","error_class":"GRPC::DeadlineExceeded","failed_at":1528390816.7237084,"retry_count":1,"retried_at":1528390938.3784637},"jobstr":"{\"class\":\"Gitlab::GithubImport::Stage::ImportRepositoryWorker\",\"args\":[232],\"retry\":5,\"queue\":\"github_importer:github_import_stage_import_repository\",\"queue_namespace\":\"github_importer\",\"dead\":false,\"jid\":\"544a8c41fcc0a0611ad32bf7\",\"created_at\":1528390714.5787973,\"enqueued_at\":1528390994.2521996,\"error_message\":\"4:Deadline Exceeded\",\"error_class\":\"GRPC::DeadlineExceeded\",\"failed_at\":1528390816.7237084,\"retry_count\":1,\"retried_at\":1528390938.3784637}"}
2018-06-07_17:04:55.13425 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/github_import/importer/repository_importer.rb:20:in `import_wiki?'
2018-06-07_17:04:55.13426 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/github_import/importer/repository_importer.rb:41:in `execute'
2018-06-07_17:04:55.13432 /opt/gitlab/embedded/service/gitlab-rails/app/workers/gitlab/github_import/stage/import_repository_worker.rb:22:in `import'
2018-06-07_17:04:55.13434 /opt/gitlab/embedded/service/gitlab-rails/app/workers/concerns/gitlab/github_import/stage_methods.rb:18:in `try_import'
2018-06-07_17:04:55.13435 /opt/gitlab/embedded/service/gitlab-rails/app/workers/concerns/gitlab/github_import/stage_methods.rb:12:in `perform'