Skip to content

Increase timeout for importing Jira issues.

Summary

While using Jira import. Customer is experiencing timeout while importing Jira issues. From our discussion on slack.. It seems like the timeout is hard-coded and cannot be configured.

Summary of findings.

Zendesk

https://gitlab.zendesk.com/agent/tickets/266984

https://gitlab.zendesk.com/agent/tickets/275820

Steps to reproduce

Setup Jira issue and import a large project.

Example Project

What is the current bug behavior?

What is the expected correct behavior?

Relevant logs and/or screenshots

{
  "severity": "ERROR",
  "time": "2022-02-15T13:40:31.344Z",
  "correlation_id": "01FVYSSXJG5EYXC9MKZN6DCF9C",
  "exception.class": "JIRA::HTTPError",
  "exception.message": "{\"message\":\"null for uri: https://xxxx/rest/api/2/label?maxResults=500&startAt=0\",\"status-code\":404}",
  "exception.backtrace": [
    "lib/gitlab/jira/http_client.rb:15:in `request'",
    "lib/gitlab/jira_import/labels_importer.rb:42:in `process_jira_page'",
    "lib/gitlab/jira_import/labels_importer.rb:32:in `block in import_jira_labels'",
    "lib/gitlab/jira_import/labels_importer.rb:31:in `loop'",
    "lib/gitlab/jira_import/labels_importer.rb:31:in `import_jira_labels'",
    "lib/gitlab/jira_import/labels_importer.rb:17:in `execute'",
    "app/workers/gitlab/jira_import/stage/import_labels_worker.rb:12:in `import'",
    "app/workers/concerns/gitlab/jira_import/import_worker.rb:21:in `perform'",
    "lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:24:in `call'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:58:in `perform'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'",
    "lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'",
    "lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `block in call'",
    "lib/gitlab/application_context.rb:74:in `block in use'",
    "lib/gitlab/application_context.rb:74:in `use'",
    "lib/gitlab/application_context.rb:27:in `with_context'",
    "lib/gitlab/sidekiq_middleware/worker_context/server.rb:15:in `call'",
    "lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'",
    "lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'",
    "lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'",
    "lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'",
    "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'",
    "lib/gitlab/with_request_store.rb:17:in `enabling_request_store'",
    "lib/gitlab/with_request_store.rb:10:in `with_request_store'",
    "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/arguments_logger.rb:8:in `call'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:66:in `block in call'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:89:in `block in instrument'",
    "lib/gitlab/metrics/background_transaction.rb:33:in `run'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:89:in `instrument'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:65:in `call'",
    "lib/gitlab/sidekiq_middleware/monitor.rb:8:in `block in call'",
    "lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'",

This later results to

{
  "severity": "ERROR",
  "time": "2022-02-15T13:41:12.307Z",
  "correlation_id": "01FVYSSXJG5EYXC9MKZN6DCF9C",
  "exception.class": "Net::ReadTimeout",
  "exception.message": "Net::ReadTimeout with #<TCPSocket:(closed)>",
  "exception.backtrace": [
    "lib/gitlab/http.rb:56:in `perform_request'",
    "lib/gitlab/jira/http_client.rb:38:in `public_send'",
    "lib/gitlab/jira/http_client.rb:38:in `make_request'",
    "lib/gitlab/jira/http_client.rb:13:in `request'",
    "lib/gitlab/jira_import/issues_importer.rb:87:in `fetch_issues'",
    "lib/gitlab/jira_import/issues_importer.rb:30:in `import_issues'",
    "lib/gitlab/jira_import/issues_importer.rb:22:in `execute'",
    "app/workers/gitlab/jira_import/stage/import_issues_worker.rb:12:in `import'",
    "app/workers/concerns/gitlab/jira_import/import_worker.rb:21:in `perform'",
    "lib/gitlab/database/load_balancing/sidekiq_server_middleware.rb:24:in `call'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/strategies/until_executing.rb:16:in `perform'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/duplicate_job.rb:58:in `perform'",
    "lib/gitlab/sidekiq_middleware/duplicate_jobs/server.rb:8:in `call'",
    "lib/gitlab/sidekiq_middleware/worker_context.rb:9:in `wrap_in_optional_context'",
    "lib/gitlab/sidekiq_middleware/worker_context/server.rb:17:in `block in call'",
    "lib/gitlab/application_context.rb:74:in `block in use'",
    "lib/gitlab/application_context.rb:74:in `use'",
    "lib/gitlab/application_context.rb:27:in `with_context'",
    "lib/gitlab/sidekiq_middleware/worker_context/server.rb:15:in `call'",
    "lib/gitlab/sidekiq_status/server_middleware.rb:7:in `call'",
    "lib/gitlab/sidekiq_versioning/middleware.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/admin_mode/server.rb:14:in `call'",
    "lib/gitlab/sidekiq_middleware/instrumentation_logger.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/batch_loader.rb:7:in `call'",
    "lib/gitlab/sidekiq_middleware/extra_done_log_metadata.rb:7:in `call'",
    "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:10:in `block in call'",
    "lib/gitlab/with_request_store.rb:17:in `enabling_request_store'",
    "lib/gitlab/with_request_store.rb:10:in `with_request_store'",
    "lib/gitlab/sidekiq_middleware/request_store_middleware.rb:9:in `call'",
    "lib/gitlab/sidekiq_middleware/arguments_logger.rb:8:in `call'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:66:in `block in call'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:89:in `block in instrument'",
    "lib/gitlab/metrics/background_transaction.rb:33:in `run'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:89:in `instrument'",
    "lib/gitlab/sidekiq_middleware/server_metrics.rb:65:in `call'",

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

```
System information
System:
Proxy: PROD_GITLAB_PAGES_SERVICE_PORT_HTTP_GITLAB_PAGES_PROXY: 8090
Current User: git
Using RVM: no
Ruby Version: 2.7.4p191
Gem Version: 3.2.19
Bundler Version:2.2.19
Rake Version: 13.0.6
Redis Version: unknown
Git Version: unknown
Sidekiq Version:6.2.2
Go Version: unknown

GitLab information
Version: 14.5.0-ee
Revision: 490a8efda84
Directory: /srv/gitlab
DB Adapter: PostgreSQL
DB Version: 12.4
URL: https://xxxx
HTTP Clone URL: https://xxxx/some-group/some-project.git
SSH Clone URL: git@xxxxx:some-group/some-project.git
Elasticsearch: yes
Geo: no
Using LDAP: no
Using Omniauth: yes
Omniauth Providers: saml

GitLab Shell
Version: 13.22.0
Repository storage paths:
- default: /var/opt/gitlab/repo
GitLab Shell path: /home/git/gitlab-shell
Git: /usr/bin/git
```

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)




Checking GitLab subtasks ...

Checking GitLab Shell ...

GitLab Shell: ... GitLab Shell version >= 13.22.0 ? ... OK (13.22.0)
Running /home/git/gitlab-shell/bin/check
gitlab-shell self-check failed
Try fixing it:
Make sure GitLab is running;
Check the gitlab-shell configuration file:
sudo -u git -H editor /home/git/gitlab-shell/config.yml
Please fix the error above and rerun the checks.

Checking GitLab Shell ... Finished

Checking Gitaly ...

Gitaly: ... default ... OK

Checking Gitaly ... Finished

Checking Sidekiq ...

Sidekiq: ... Running? ... no
Try fixing it:
sudo -u git -H RAILS_ENV=production bin/background_jobs start
For more information see:
doc/install/installation.md in section "Install Init Script"
see log/sidekiq.log for possible errors
Please fix the error above and rerun the checks.
Checking Sidekiq ... Finished

Checking Incoming Email ...

Incoming Email: ... Reply by email is disabled in config/gitlab.yml

Checking Incoming Email ... Finished

Checking LDAP ...

LDAP: ... LDAP is disabled in config/gitlab.yml

Checking LDAP ... Finished

Checking GitLab App ...

Git configured correctly? ... no
Trying to fix error automatically. ...Failed
Try fixing it:
sudo -u git -H "/usr/bin/git" config --global core.autocrlf "input"
For more information see:
doc/install/installation.md in section "GitLab"
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet)
Systemd unit files or init script exist? ... no
Try fixing it:
Install the Service
For more information see:
doc/install/installation.md in section "Install the Service"
Please fix the error above and rerun the checks.
Systemd unit files or init script up-to-date? ... can't check because of previous errors
Projects have namespace: ...
5/3 ... yes
5/4 ... yes
(omitted similar lines…)
5/5902 ... yes
2131/5903 ... yes
308/5904 ... yes
600/5905 ... yes
1227/5906 ... yes
218/5907 ... yes
308/5908 ... yes
411/5909 ... yes
696/5932 ... yes
Redis version >= 5.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.4)
Git version >= 2.33.0 ? ... no
Your git bin path is "/usr/bin/git"
Try fixing it:
Update your git to a version >= 2.33.0 from Unknown
Please fix the error above and rerun the checks.
Git user has default SSH configuration? ... yes
Active users: ... xxx
Is authorized keys file accessible? ... skipped (authorized keys not enabled)
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x (6.4 - 6.x deprecated to be removed in 13.8)? ... yes (7.12.0)

Checking GitLab App ... Finished

Checking GitLab subtasks ... Finished







(we will only investigate if the tests are passing)

Possible fixes

Increase the Jira client read timeout limit, but only within the Jira issue importer, rather than all Jira API requests (https://gitlab.com/gitlab-org/gitlab/-/issues/343174).

Edited by Luke Duncalfe