Rouge::Lexer.guess can timeout and cause 500 errors

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

When viewing certain files in the UI, or searching via Advanced Search, the Rouge::Lexer.guess can take a long time, leading to timeouts and 500 errors.

This was the case with a particular customer. Zendesk ticket refers (internal).

Steps to reproduce

  1. Add bootstrap.bundle.js.map from Bootstrap to a repository
  2. Enable Advanced Search indexing
  3. View file in UI
  4. Search for file in Advanced Search

Example Project

What is the current bug behavior?

Rack::Timeout::RequestTimeoutException returned. duration_s can reach 60 seconds, and cpu_s is the majority of this time. This results in 500 errors.

What is the expected correct behavior?

Improve performance of syntax highlighting to avoid timeouts

Relevant logs and/or screenshots

"exception.class":"Rack::Timeout::RequestTimeoutException","exception.message":"Request ran for longer than 60000ms ","exception.backtrace":["app/presenters/blob_presenter.rb:223:in `detect_language'","app/presenters/blob_presenter.rb:51:in `blob_language'","app/presenters/blob_presenter.rb:19:in `highlight_and_trim'","app/views/search/results/_blob_highlight.html.haml:7","app/views/search/results/_blob_data.html.haml:12","app/views/search/results/_blob.html.haml:8","app/helpers/application_helper.rb:14:in `render_if_exists'","app/views/search/_results_list.html.haml:18","app/views/search/_results.html.haml:9","app/views/search/show.html.haml:24","app/controllers/application_controller.rb:142:in `render'","app/controllers/application_controller.rb:573:in `block in allow_gitaly_ref_name_caching'","lib/gitlab/gitaly_client.rb:352:in `allow_ref_name_caching'","app/controllers/application_controller.rb:572:in `allow_gitaly_ref_name_caching'","ee/lib/gitlab/ip_address_state.rb:10:in `with'","ee/app/controllers/ee/application_controller.rb:46:in `set_current_ip_address'","app/controllers/application_controller.rb:524:in `set_current_admin'","lib/gitlab/session.rb:11:in `with_session'","app/controllers/application_controller.rb:515:in `set_session_storage'","lib/gitlab/i18n.rb:107:in `with_locale'","lib/gitlab/i18n.rb:113:in `with_user_locale'","app/controllers/application_controller.rb:506:in `set_locale'","app/controllers/application_controller.rb:499:in `set_current_context'","lib/gitlab/metrics/elasticsearch_rack_middleware.rb:16:in `call'","lib/gitlab/middleware/memory_report.rb:13:in `call'","lib/gitlab/middleware/speedscope.rb:13:in `call'","lib/gitlab/database/load_balancing/rack_middleware.rb:23:in `call'","lib/gitlab/middleware/rails_queue_duration.rb:33:in `call'","lib/gitlab/metrics/rack_middleware.rb:16:in `block in call'","lib/gitlab/metrics/web_transaction.rb:46:in `run'","lib/gitlab/metrics/rack_middleware.rb:16:in `call'","lib/gitlab/jira/middleware.rb:19:in `call'","lib/gitlab/middleware/go.rb:20:in `call'","lib/gitlab/etag_caching/middleware.rb:21:in `call'","lib/gitlab/middleware/query_analyzer.rb:11:in `block in call'","lib/gitlab/database/query_analyzer.rb:37:in `within'","lib/gitlab/middleware/query_analyzer.rb:11:in `call'","lib/gitlab/middleware/multipart.rb:173:in `call'","lib/gitlab/middleware/read_only/controller.rb:50:in `call'","lib/gitlab/middleware/read_only.rb:18:in `call'","lib/gitlab/middleware/same_site_cookies.rb:27:in `call'","lib/gitlab/middleware/basic_health_check.rb:25:in `call'","lib/gitlab/middleware/handle_malformed_strings.rb:21:in `call'","lib/gitlab/middleware/handle_ip_spoof_attack_error.rb:25:in `call'","lib/gitlab/middleware/request_context.rb:21:in `call'","lib/gitlab/middleware/webhook_recursion_detection.rb:15:in `call'","config/initializers/fix_local_cache_middleware.rb:11:in `call'","lib/gitlab/middleware/compressed_json.rb:37:in `call'","lib/gitlab/middleware/rack_multipart_tempfile_factory.rb:19:in `call'","lib/gitlab/middleware/sidekiq_web_static.rb:20:in `call'","lib/gitlab/metrics/requests_rack_middleware.rb:79:in `call'","lib/gitlab/middleware/release_env.rb:13:in `call'"]

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

Workaround

Add the extension to .gitattributes file.

Example:

*.js.map gitlab-language=text

Possible fixes

Edited by 🤖 GitLab Bot 🤖