SSL Error in Web Hook (Sidekiq)
I am see an SSL error in /var/log/gitlab/sidekiq/current. I believe the error to be from a web hook trying to send a message to an internal service using a self-signed CA. I've added the CA to the system trusted CA bundle and, for example, curl can now connect to the web hook fine. However, I am still seeing the error from Sidekiq. Is there a way to configure Sidekiq/Ruby to use the system trusted CA bundle? Does GitLab uses its own CA bundle?
GitLab was installed using the Omnibus package on to RHEL 6.5.
2015-08-19_18:31:06.00765 2015-08-19T18:31:06.006Z 4285 TID-6ssoo ProjectServiceWorker JID-8f823bd40ab893bd81f19b50 INFO: start
2015-08-19_18:31:06.03988 2015-08-19T18:31:06.039Z 4285 TID-6ssoo ProjectServiceWorker JID-8f823bd40ab893bd81f19b50 INFO: fail: 0.033 sec
2015-08-19_18:31:06.04093 2015-08-19T18:31:06.040Z 4285 TID-6ssoo WARN: {"retry"=>true, "queue"=>"project_web_hook", "class"=>"ProjectServiceWorker", "args"=>[34, {"object_kind"=>"push", "before"=>"ad685abbfde56f98164ef25156b54673e8ea23d4", "after"=>"0000000000000000000000000000000000000000", "ref"=>"refs/heads/REDACTED", "checkout_sha"=>nil, "message"=>nil, "user_id"=>2, "user_name"=>"REDACTED", "user_email"=>"REDACTED", "project_id"=>2, "repository"=>{"name"=>"REDACTED", "url"=>"git@REDACTED:REDACTED.git", "description"=>"", "homepage"=>"https://REDACTED:8843/REDACTED", "git_http_url"=>"https://REDACTED:8843/REDACTED.git", "git_ssh_url"=>"git@REDACTED:REDACTED.git", "visibility_level"=>0}, "commits"=>[], "total_commits_count"=>0}], "jid"=>"8f823bd40ab893bd81f19b50", "enqueued_at"=>1439572838.5764277, "error_message"=>"SSL_connect returned=1 errno=0 state=error: certificate verify failed", "error_class"=>"OpenSSL::SSL::SSLError", "failed_at"=>1439572841.6877923, "retry_count"=>19, "retried_at"=>1440009066.0394087}
2015-08-19_18:31:06.04101 2015-08-19T18:31:06.040Z 4285 TID-6ssoo WARN: SSL_connect returned=1 errno=0 state=error: certificate verify failed
2015-08-19_18:31:06.04112 2015-08-19T18:31:06.041Z 4285 TID-6ssoo WARN: /opt/gitlab/embedded/lib/ruby/2.1.0/net/http.rb:923:in `connect'
2015-08-19_18:31:06.04114 /opt/gitlab/embedded/lib/ruby/2.1.0/net/http.rb:923:in `block in connect'
2015-08-19_18:31:06.04116 /opt/gitlab/embedded/lib/ruby/2.1.0/timeout.rb:76:in `timeout'
2015-08-19_18:31:06.04118 /opt/gitlab/embedded/lib/ruby/2.1.0/net/http.rb:923:in `connect'
2015-08-19_18:31:06.04118 /opt/gitlab/embedded/lib/ruby/2.1.0/net/http.rb:863:in `do_start'
2015-08-19_18:31:06.04118 /opt/gitlab/embedded/lib/ruby/2.1.0/net/http.rb:852:in `start'
2015-08-19_18:31:06.04119 /opt/gitlab/embedded/lib/ruby/2.1.0/net/http.rb:1375:in `request'
2015-08-19_18:31:06.04119 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/httparty-0.13.3/lib/httparty/request.rb:98:in `perform'
2015-08-19_18:31:06.04119 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/httparty-0.13.3/lib/httparty.rb:539:in `perform_request'
2015-08-19_18:31:06.04119 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/httparty-0.13.3/lib/httparty.rb:521:in `head'
2015-08-19_18:31:06.04120 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/httparty-0.13.3/lib/httparty.rb:605:in `head'
2015-08-19_18:31:06.04120 /opt/gitlab/embedded/service/gitlab-rails/app/models/project_services/issue_tracker_service.rb:84:in `execute'
2015-08-19_18:31:06.04120 /opt/gitlab/embedded/service/gitlab-rails/app/workers/project_service_worker.rb:8:in `perform'
2015-08-19_18:31:06.04121 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:75:in `execute_job'
2015-08-19_18:31:06.04121 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:52:in `block (2 levels) in process'
2015-08-19_18:31:06.04121 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:127:in `call'
2015-08-19_18:31:06.04121 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:127:in `block in invoke'
2015-08-19_18:31:06.04122 /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/sidekiq_middleware/memory_killer.rb:17:in `call'
2015-08-19_18:31:06.04122 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
2015-08-19_18:31:06.04122 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidetiq-0.6.3/lib/sidetiq/middleware/history.rb:8:in `call'
2015-08-19_18:31:06.04122 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
2015-08-19_18:31:06.04123 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/server/active_record.rb:6:in `call'
2015-08-19_18:31:06.04125 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
2015-08-19_18:31:06.04126 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/server/retry_jobs.rb:74:in `call'
2015-08-19_18:31:06.04126 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
2015-08-19_18:31:06.04126 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/server/logging.rb:11:in `block in call'
2015-08-19_18:31:06.04127 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/logging.rb:22:in `with_context'
2015-08-19_18:31:06.04127 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/server/logging.rb:7:in `call'
2015-08-19_18:31:06.04127 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:129:in `block in invoke'
2015-08-19_18:31:06.04127 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:132:in `call'
2015-08-19_18:31:06.04128 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/middleware/chain.rb:132:in `invoke'
2015-08-19_18:31:06.04128 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:51:in `block in process'
2015-08-19_18:31:06.04128 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:98:in `stats'
2015-08-19_18:31:06.04129 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/sidekiq-3.3.0/lib/sidekiq/processor.rb:50:in `process'
2015-08-19_18:31:06.04129 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `public_send'
2015-08-19_18:31:06.04129 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in `dispatch'
2015-08-19_18:31:06.04129 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:122:in `dispatch'
2015-08-19_18:31:06.04130 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in `block in invoke'
2015-08-19_18:31:06.04130 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in `block in task'
2015-08-19_18:31:06.04130 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in `block in task'
2015-08-19_18:31:06.04132 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in `block in initialize'
2015-08-19_18:31:06.04132 /opt/gitlab/embedded/service/gem/ruby/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in `block in create'