A lot of ERROR -- : app error: BADMTHD, accepted HTTP methods are OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, CONNECT
Hi,
i can see a lot (2 to 3 per second) of errors in the unicorn error log:
ERROR -- : app error: BADMTHD, accepted HTTP methods are OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, CONNECT, PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, LOCK, UNLOCK, VERSION-CONTROL, REPORT, CHECKOUT, CHECKIN, UNCHECKOUT, MKWORKSPACE, UPDATE, LABEL, MERGE, BASELINE-CONTROL, MKACTIVITY, ORDERPATCH, ACL, SEARCH, MKCALENDAR, and PATCH (ActionController::UnknownHttpMethod)
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/http/request.rb:344:in `check_method'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/http/request.rb:105:in `request_method'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:51:in `started_request_message'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:37:in `block in call_app'
ERROR -- : /opt/gitlab/embedded/lib/ruby/2.4.0/logger.rb:463:in `add'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/logger.rb:88:in `add'
ERROR -- : /opt/gitlab/embedded/lib/ruby/2.4.0/logger.rb:525:in `info'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:37:in `call_app'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:20:in `block in call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/tagged_logging.rb:68:in `block in tagged'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/tagged_logging.rb:26:in `tagged'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/activesupport-4.2.10/lib/active_support/tagged_logging.rb:68:in `tagged'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/rack/logger.rb:20:in `call'
ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/request_context.rb:18:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/request_store-1.3.1/lib/request_store/middleware.rb:9:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/actionpack-4.2.10/lib/action_dispatch/middleware/request_id.rb:21:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/methodoverride.rb:22:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/runtime.rb:18:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/lock.rb:17:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/sendfile.rb:113:in `call'
ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/metrics/requests_rack_middleware.rb:27:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/sentry-raven-2.7.2/lib/raven/integrations/rack.rb:51:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/engine.rb:518:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/application.rb:165:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/railtie.rb:194:in `public_send'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/railties-4.2.10/lib/rails/railtie.rb:194:in `method_missing'
ERROR -- : /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/middleware/release_env.rb:10:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/urlmap.rb:66:in `block in call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/urlmap.rb:50:in `each'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/rack-1.6.10/lib/rack/urlmap.rb:50:in `call'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:562:in `process_client'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-worker-killer-0.4.4/lib/unicorn/worker_killer.rb:52:in `process_client'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:658:in `worker_loop'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:519:in `maintain_worker_count'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:283:in `join'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/bin/unicorn:126:in `<top (required)>'
ERROR -- : /opt/gitlab/embedded/bin/unicorn:23:in `load'
ERROR -- : /opt/gitlab/embedded/bin/unicorn:23:in `<top (required)>'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/cli/exec.rb:74:in `load'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/cli/exec.rb:74:in `kernel_load'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/cli/exec.rb:28:in `run'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/cli.rb:424:in `exec'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor/invocation.rb:126:in `invoke_command'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor.rb:387:in `dispatch'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/cli.rb:27:in `dispatch'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/vendor/thor/lib/thor/base.rb:466:in `start'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/cli.rb:18:in `start'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/exe/bundle:30:in `block in <top (required)>'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/lib/bundler/friendly_errors.rb:124:in `with_friendly_errors'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.16.2/exe/bundle:22:in `<top (required)>'
ERROR -- : /opt/gitlab/embedded/bin/bundle:23:in `load'
ERROR -- : /opt/gitlab/embedded/bin/bundle:23:in `<main>'
Anyone getting the same errors? We are running on (ce docker image):
GitLab 11.4.5 (gitlab-ce@f5536c6121077ea5d70d488e40af78acccc5cd8f) GitLab Shell 8.3.3 GitLab Workhorse v7.0.0 GitLab API v4 Ruby 2.4.5p335 Rails 4.2.10 postgresql 9.6.6
and see degraded performance at the same time. Any ideas what is causing these errors, or what's the result of these errors?
Because it says
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-worker-killer-0.4.4/lib/unicorn/worker_killer.rb:52:in `process_client'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:658:in `worker_loop'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:519:in `maintain_worker_count'
ERROR -- : /opt/gitlab/embedded/lib/ruby/gems/2.4.0/gems/unicorn-5.1.0/lib/unicorn/http_server.rb:283:in `join'
i think maybe there are problems in spawning new unicorn workers? Any ideas?